L4 - Black Box and Glass Box Testing Summary

Black Box Testing Principles

• Goal: Design strong test suites that cover core functionality and component integration, are easy to maintain, and provide reliable, deterministic feedback.

• Methods: Equivalence Class Partitioning (ECP) and Boundary Value Analysis (BVA) are used together.

Equivalence Class Partitioning (ECP)

• Purpose: Systematically cover the unbounded input/output space and reduce redundant tests.

• Process:

  1. Partition the set of possible inputs/outputs ($D$) into equivalence classes ($A$, $B$, $C$) that:

     • Characterize sets with something in common.

     • Cover $D$ ($A \cup B \cup C = D$).

     • Are disjoint ($A \cap B \cap C = \emptyset$).

  2. Test one representative input from each class.

• Input Partitioning Example (abs(x)): Classes for negative, zero, and positive numbers.

• Output Partitioning: Requires identifying inputs that produce outputs in each defined output partition; sometimes better at finding bugs than input partitioning alone.

Boundary Value Analysis (BVA)

• Purpose: Find common errors in edge cases, often caused by off-by-one errors (e.g., < vs. _).

• Process: Test values on either side of each partition boundary (in-bound, on-bound, and out-of-bound values).

ECP and BVA Application

• When to Apply: Most appropriate for fully-specified, client-facing functions.

• Benefits: Easy to start, finds high-severity bugs (user perspective), less overfitting to implementation, parallelizes development and testing.

• Drawbacks: Relies on well-defined specifications, assertion strength depends on input-output relationship, can make fault localization difficult.

Glass Box Testing

• Purpose: Evaluate the effectiveness and completeness of a test suite against the implementation itself.

• Contrast with Black Box: Black Box tests against specifications (prior to implementation), Glass Box tests against implementation (after implementation).

Measures:

• Test Coverage: Quantifies how much code is exercised by tests.

  • Line Coverage: Measures the percentage of lines executed.

  Ex:

  

  • Statement Coverage: Assesses whether each statement was executed, ensuring critical paths are validated.

  Ex:

  

• • Branch Coverage: Measures if all branches (true/false paths of an if statement) are executed. (if 2 if statements, doesn’t have to be both true and false to get 100%)

  Ex:

  

  • Path Coverage: Measures if all possible paths through the code are executed; often infeasible due to complexity.

  Ex:

• Mutation Testing: Assesses if tests can detect small, randomly inserted code changes (typos).