Chapter 9: Testing

Software Testing

A process in which you execute your program using data that simulates user inputs. ​

The 2 Causes of Programming Bugs

• Programming Errors

• Understanding Errors

Programming Errors

You have accidentally included faults in your program code.

Understanding Errors

You have misunderstood or have been unaware of some of the details of what the program is supposed to do.

The 4 Testing Types

• Functional

• User

• Performance and Load

• Security

Functional Testing

Test the functionality of the overall system.

User Testing

Test that the software product is useful to and usable by end-users.

Performance and Load Testing

Test that the software works quickly and can handle the expected load placed on the system by its users.

Security Testing

Test that the software maintains its integrity and can​ protect user information from theft and damage; aims to find vulnerabilities that may be exploited by an attacker and to provide convincing evidence that the system is sufficiently secure.

Functional Testing Cycle

1. Unit Testing

2. Feature Testing

3. System Testing

4. Release Testing

Unit Testing

Test program units in isolation.

Feature Testing

Test code units that have been integrated to create features.

System Testing

Check that there are no unexpected interactions between the features in the system; testing the system as a whole, rather than the individual system features. ​

Release Testing

A type of system testing where a system that’s intended for release to customers is tested; the system is packaged for release to customers and the release is tested to check that it operates as expected

Code Unit

Anything that has a clearly defined responsibility.

Equivalence Partition

A set of inputs; used during unit testing.

Incorrectness Partitions

Sets of inputs that are deliberately incorrect; used during unit testing.

Unit Testing Guidlines

• Test Edge Cases

• Force Errors

• Fill Buffers

• Repeat Yourself

• Overflow and Underflow

• Don’t Forget Null and Zero

• Keep Count

• One is Different

Fill Buffers

Choose test inputs that cause all input buffers to overflow.​

Keep Count

When dealing with lists and list transformations, keep​ count of the number of elements in each list and check​ that these are consistent after each transformation.

One is Different

If your program deals with sequences, always test with​ sequences that have a single value.

The 2 Types of Feature Tests

• Interaction Tests

• Usefulness Tests

Interaction Tests

These test the interactions between the units that implement the feature.

Usefulness Tests

These test that the feature implements what users are likely to want.

The 3 Parts of Automated Testing

• Arrange

• Action

• Assert

Arrange

Set up the system to run the test.

Action

Call the unit that is being tested with the test parameters. ​

Assert

Make an assertion about what should hold if the unit being tested has executed successfully.

The Test Pyramid (from Top to Bottom)

• System tests

• Feature tests

• Unit tests

Test-Driven Development (TDD)

An approach to program development that is based around the general idea that you should write an executable test or tests for code that you are writing before you write the code. ​

Test Driven Development Cycle

1. Identify New Functionality

2. Identify Partial Implementation of Functionality

3. Write Code Stub that will Fail Test

4. Run All Automated Tests

5. Implement Code that Should Cause Failing Test to Pass

6. Run All Automated Tests

7. Refactor Code If Required

Identify Partial Implementation

Break down the implementation of the functionality required into smaller mini-units.

Write Mini-Unit Tests

Write one or more automated tests for the mini-unit​ that you have chosen for implementation.

Write Code Stub that will Fail Test

Write incomplete code that will be called to​ implement the mini-unit.

Benefits of Test-Driven Development

• A systematic approach to testing in which tests are clearly linked to sections of the program code.

• Should be possible to understand what the program does by reading the tests. ​

• Debugging is simplified.

• Leads to simpler code.

Disadvantages of TDD

• Discourages radical program change.

• Focus is shifted towards tests rather than the problem trying to be solved.

• Focus is shifted towards implementation details rather than the programming problem.

• It is hard to write “bad data” tests.

Risked-Based Security Testing

Involves identifying common risks and developing tests to demonstrate that the system protects itself from these risks. Also consists of risk analysis.

Risk Analysis

Consists of analyzing risks to assess how they might arise and developing tests to check some of these possibilities.

Code Reviews

Involve one or more people examining the code to check for errors and anomalies and discussing issues with the developer. ​

Code Review Activities

• Set Up Review

• Prepare Code

• Distribute/Code Tests

• Check Code

• Write Review Report

• Discussion

• Make To-Do List

• Make Code Changes

Set Up Review

The programmer contacts a reviewer and arranges a review date.

Prepare Code

The programmer collects the code and tests for review and annotates them with information for the reviewer about the intended purpose of the code and tests.

Distribute Code/Tests

The programmer sends code and tests to the reviewer.

Check Code

The reviewer systematically checks the code and tests against their understanding of what they are supposed to do.

Write Review Report

The reviewer annotates the code and tests with a report of the issues to be discussed at the review meeting.

Discussion

The reviewer and programmer discuss the issues and agree on the actions to resolve these.

Make To-Do List

The programmer documents the outcome of the review as a to-do list and shares this with the reviewer.

Make Code Changes

The programmer modifies the code and tests to address the issues raised in the review.​