Chapter 5: Software Testing Techniques and Strategies

Testability

The degree to which a software system can be effectively tested.

Operability

It operates cleanly.

Observability

The results of each test case are readily observed.

Controllability

The degree to which testing can be automated and optimized.

Decomposability

Testing can be targeted.

Simplicity

Reduce complex architecture and logic to simplify tests.

Stability

Few changes are requested during testing.

Understandability

Of the design.

Good Test

A good test has a high probability of finding an error.

Redundancy in Testing

A good test is not redundant.

Best of Breed Test

A good test should be 'best of breed'.

Test Complexity

A good test should be neither too simple nor too complex.

Internal Testing

Tests conducted to ensure that internal operations are performed according to specifications.

External Testing

Tests conducted that demonstrate each function is fully operational while searching for errors.

Test Case Design

The objective is to uncover errors in a complete manner with a minimum of effort and time.

Exhaustive Testing

If we execute one test per millisecond, it would take 3,170 years to test this program.

Selective Testing

Testing selected paths.

Black-Box Testing

Testing methods that focus on input and output without knowledge of internal workings.

White-Box Testing

Our goal is to ensure that all statements and conditions have been executed at least once.

Cyclomatic Complexity

Number of simple decisions + 1 or number of enclosed areas + 1.

Basis Path Testing

A method that derives independent paths based on cyclomatic complexity.

Independent Paths

Paths derived from the cyclomatic complexity to exercise during testing.

Graph Matrix

A square matrix whose size is equal to the number of nodes on a flow graph.

Graph Matrix

A powerful tool for evaluating program control structure during testing by adding a link weight to each matrix entry.

Condition Testing

A test case design method that exercises the logical conditions contained in a program module.

Data Flow Testing

Selects test paths of a program according to the locations of definitions and uses of variables in the program.

DEF(S)

{X | statement S contains a definition of X}.

USE(S)

{X | statement S contains a use of X}.

Definition-Use (DU) Chain

Of variable X is of the form [X, S, S'], where S and S' are statement numbers, X is in DEF(S) and USE(S'), and the definition of X in statement S is live at statement S'.

Loop Testing

A testing method that includes various types of loops such as simple, nested, concatenated, and unstructured loops.

Simple Loops Minimum Conditions

1. Skip the loop entirely; 2. Only one pass through the loop; 3. Two passes through the loop; 4. m passes through the loop where m < n; 5. (n-1), n, and (n+1) passes through the loop where n is the maximum number of allowable passes.

Nested Loops Testing

Start at the innermost loop, set all outer loops to their minimum iteration parameter values, and test the innermost loop while holding outer loops at their minimum values.

Concatenated Loops

If the loops are independent of one another, treat each as a simple loop; else treat as nested loops.

Black-Box Testing

A testing method that focuses on requirements, output, events, and input without knowledge of internal code structure.

Functional Validity Testing

How is functional validity tested?

System Behavior Testing

How is system behavior and performance tested?

Input Classes for Test Cases

What classes of input will make good test cases?

Sensitivity to Input Values

Is the system particularly sensitive to certain input values?

Boundary Isolation

How are the boundaries of a data class isolated?

Data Rates and Volume

What data rates and data volume can the system tolerate?

Data Combinations Effect

What effect will specific combinations of data have on system operation?

Graph-Based Methods

Methods to understand the objects modeled in software and the relationships that connect these objects.

Equivalence Partitioning

A testing technique that divides input data into valid and invalid partitions to reduce the number of test cases.

Sample Equivalence Classes

Valid data includes user supplied commands, responses to system prompts, file names, computational data, physical parameters, bounding values, initiation values, output data formatting, and graphical data.

Invalid Data

Data outside bounds of the program, physically impossible data, or proper value supplied in the wrong place.

Boundary Value Analysis

A testing technique that focuses on the values at the edges of input domains.

Comparison Testing

Used in critical reliability situations where separate teams develop independent versions of an application to ensure identical output.

Orthogonal Array Testing

Used when the number of input parameters is small and the values that each parameter may take are clearly bounded.

Software Testing Patterns

Described similarly to design patterns, providing techniques for testing software from the user's point of view.

Scenario Testing Pattern

A technique for exercising the software from the user's point of view to determine if it meets user visible requirements.