CH 4 Software Test Analysis

Test Analysis

The phase where the test basis is analyzed to identify test conditions.

Test Basis

Any project document used as the basis for testing (requirements, user stories, specifications, etc.).

Test Oracle

A source used to determine expected results, such as requirements, SMEs, or existing systems.

Test Condition

A testable aspect of a component or system identified as a basis for testing.

Given-When-Then Format

A structured way of writing test conditions and test cases where Given = Preconditions, When = Inputs/Actions, and Then = Expected Results/Postconditions.

Test Design

The phase where test conditions are transformed into test cases.

Test Case

A set of preconditions, inputs, actions, expected results, and postconditions developed from test conditions.

Preconditions

Conditions that must exist before a test can be executed.

Inputs

Data entered into the system during testing.

Actions

Operations performed during a test.

Expected Results

The outcomes that should occur after processing inputs.

Postconditions

Conditions that exist after a test has been completed.

Test Implementation

The phase where test procedures are created from test cases.

Test Procedure

A sequence of test cases and associated actions executed in a specific order.

Traceability

The ability to link requirements, test conditions, test cases, and test procedures together.

Purpose of Traceability

Supports impact analysis when requirements change, tracks coverage, improves reporting, and helps maintain test artifacts.

Purpose of Test Techniques

Reduce the number of tests needed, improve coverage, standardize testing, and increase efficiency.

Three Categories of Test Techniques

Black Box Testing, White Box Testing, and Experience-Based Testing.

Black Box Testing

A specification-based testing technique that evaluates system behavior without examining internal structure.

Behavioral Testing

Another term for black box testing because it focuses on system behavior.

Characteristics of Black Box Testing

Tests inputs and outputs, is independent of code structure, and verifies functionality and some non-functional requirements.

White Box Testing

A structure-based testing technique that evaluates internal structure and code coverage.

Structure-Based Testing

Another name for white box testing because it focuses on how a system is built.

Purpose of White Box Testing

Measures coverage of internal structure, often through code coverage.

Limitation of White Box Testing

Does not determine whether the system is fit for purpose.

Experience-Based Testing

Testing that relies on tester knowledge, intuition, and previous experience.

Purpose of Experience-Based Testing

Find defects missed by other techniques and support testing when time is limited.

Characteristics of Experience-Based Testing

Supplementary technique usually performed after black box and white box testing.

Equivalence Partitioning (EP)

A black box test technique that divides data into groups (partitions) expected to behave similarly.

Partition (Equivalence Partition)

A group of values expected to be processed in the same way.

Valid Partition

A group of acceptable input values.

Invalid Partition

A group of unacceptable input values that should be rejected.

Key Principle of Equivalence Partitioning

One representative value is selected from each partition.

Benefits of Equivalence Partitioning

Reduces test cases, provides measurable coverage, and avoids redundant testing.

Coverage in Equivalence Partitioning

Partitions tested divided by total partitions.

Each Choice Coverage

A coverage technique where each partition is tested at least once.

Purpose of Each Choice Coverage

Achieve coverage with the minimum number of test cases.

Error Masking

A situation where one defect hides another defect.

Preventing Error Masking

Test each invalid partition separately whenever possible.

Input Partition

A partition based on input values entered into the system.

Output Partition

A partition based on expected system outputs.

Test Analysis Creates

Test Conditions.

Test Design Creates

Test Cases.

Test Implementation Creates

Test Procedures.

Black Box Testing Focuses On

Behavior and expected outputs.

White Box Testing Focuses On

Internal structure and coverage.

Experience-Based Testing Focuses On

Tester knowledge and intuition.

Goal of Equivalence Partitioning

Reduce tests by grouping similar values.

Why Avoid Multiple Invalid Inputs in One EP Test?

To prevent error masking.

Review Culture

A people-related success factor where reviews focus on learning, quality improvement, and helping the author identify defects rather than assigning blame.

Statement Coverage

White box testing technique that measures the percentage of executable statements (lines of code) that have been executed by test cases.

Branch Coverage

White box testing technique that measures whether every possible decision outcome (true and false) has been executed.

Decision Coverage

Another name for branch coverage because it focuses on testing all outcomes of decisions.

Nested IF Statement

An IF statement contained inside another IF statement.

Statement Coverage vs Branch Coverage

Branch coverage is stronger because it tests both true and false outcomes of decisions, while statement coverage only ensures executable statements run.

Branch

A path connecting two points in a control flow diagram, often representing a decision outcome.

Loop Structure

A programming structure that repeats execution of code while a condition remains true.

WHILE Loop

A loop that continues executing as long as its condition evaluates to true.

SELECT CASE Statement

A programming structure used to select one action from multiple possible conditions, often replacing multiple IF/ELSEIF statements.

Path Coverage

White box testing technique that tests every possible path through a program.

Path Coverage Complexity

The number of required test cases grows exponentially as more decisions are added.

White Box Testing Hierarchy

Path coverage subsumes branch coverage, and branch coverage subsumes statement coverage.

Subsumes

Includes all test cases required by a lower-level coverage technique and potentially additional tests.

Why Use Statement Coverage?

It requires fewer test cases and less effort than branch coverage.

Risk of Statement Coverage

It may miss defects because not all decision outcomes are tested.

Advantage of Branch Coverage

Provides greater confidence in code quality and can detect more defects.

Control Flow Analysis

Static analysis technique that reviews code structure to identify possible execution paths without running the code.

Static Testing

Testing activities that examine work products without executing the software.

Dynamic Testing

Testing that involves executing software and comparing actual results with expected results.

Experience-Based Testing

Testing that relies on the tester's knowledge, intuition, skills, and past experience to design tests.

Purpose of Experience-Based Testing

Supplements systematic testing techniques by identifying additional test cases based on experience.

Reasons to Use Experience-Based Testing

Lack of detailed documentation, high-risk areas, historical defects, unusual value combinations, or inexperienced developers.

Three Experience-Based Test Techniques

Error guessing, exploratory testing, and checklist-based testing.

Error Guessing

Technique that uses knowledge of previous defects and experience to predict likely problem areas.

Examples of Error Guessing

Leaving mandatory fields blank, entering invalid dates, using maximum values, and entering letters in numeric fields.

Advantage of Error Guessing

Can uncover defects not found through systematic testing.

Disadvantage of Error Guessing

Heavily dependent on tester experience.

Fault Attack

Structured form of error guessing where testers identify possible failures and create tests to expose them.

Fault Attack Workshop

Collaborative session where multiple testers brainstorm possible defects and attack scenarios.

Exploratory Testing

Simultaneous learning, test design, and test execution based on exploration of the software.

When Exploratory Testing Is Useful

Limited time, poor documentation, usability testing, or investigating defects.

Advantage of Exploratory Testing

Experienced testers can quickly find significant defects.

Disadvantage of Exploratory Testing

Difficult to measure coverage and reproduce defects if notes are poor.

Heuristic Evaluation

Exploratory testing focused on evaluating usability.

Session-Based Testing

Structured exploratory testing performed during a defined time-box using a specific charter.

Session

An uninterrupted period of exploratory testing.

Time-Box

A fixed amount of time allocated to a testing session.

Charter

A documented objective or focus area for exploratory testing.

Debrief

Review meeting after a session to discuss findings and test results.

Exploratory Testing vs Ad Hoc Testing

Exploratory testing is focused and structured around a charter; ad hoc testing is unstructured.

Checklist-Based Testing

Experience-based technique that uses predefined lists of items or questions to guide testing.

Purpose of Checklists

Ensure consistency and standardization of testing activities.

Advantages of Checklist-Based Testing

Consistent quality assessment, repeatability, and easier reviews.

Disadvantages of Checklist-Based Testing

Testers may only test listed items and overlook other defects.

Possible Error Guessing Test

Leave username and password fields blank.

Boundary Value Test Example

Enter a username longer than the maximum allowed length.

Decision Table Test Example

Test combinations of valid and invalid usernames and passwords.

State Transition Test Example

Verify transitions between login screen and main menu after successful login.

SQL Injection Example

Entering malicious SQL code to bypass authentication.

Rule 1

Functional testing should be the first testing priority.