Ch8.Testing

Software Testing Overview

• Software testing is essential for ensuring programs function as intended and to find defects before deployment.

• Testing reveals errors but does not guarantee their absence.

• It forms part of a broader verification and validation (V&V) process including static validation methods.

Goals of Program Testing

Development Objectives

• Demonstrate to both developer and customer that software meets requirements.

  • Each requirement in a custom software project should have corresponding test cases.

  • For generic software, tests should cover all features and their combinations.

• Identify incorrect, undesirable system behaviors.

  • Focused on defect testing, which seeks to find operational failures like crashes and data issues.

Testing Types

Validation vs. Defect Testing

• Validation Testing: Ensures the system performs as expected with given test cases reflecting real usage.

• Defect Testing: Uses obscure test cases to expose defects, which may not reflect typical system use.

Testing Processes

Goals

• Validation Testing: Validate that software meets requirements.

  • Successful tests demonstrate intended system operation.

• Defect Testing: Identify faults or discrepancies in software behavior.

  • Successful tests highlight incorrect operations exposing defects.

Input-Output Testing Model

• Inputs are artificial data used in testing.

• Outputs reveal whether defects exist based on the program's response to the tests.

Verification and Validation (V&V)

Definitions

• Verification: Ensures the product is built correctly, conforming to specifications.

• Validation: Confirms the product meets user needs and intended use.

V&V Confidence Levels

• Confidence in a system's fitness is influenced by:

  • Software criticality.

  • User expectations.

  • Marketing urgency overriding defect discovery.

Inspections and Testing

Inspections

• Focus on analyzing static representations to detect defects before implementation.

• Effective for catching errors without execution.

Testing

• Dynamic verification through system execution with test data.

• Allows for observation of operational behavior.

Benefits of Inspections

• Inspecting uncover hidden errors not visible during testing.

• Incomplete systems can be analyzed without requiring extensive test setups.

• Inspections can also assess broader quality attributes beyond just defect detection.

Software Testing Process

Stages

• Development testing

  • Includes unit, component, and system testing to find bugs early in development.

• Release testing

  • Conducted by separate teams on fully integrated systems before user release.

• User testing

  • Involves actual users testing software in real work environments.

Unit Testing

Definition

• Tests individual components or units of code in isolation.

• Primarily focuses on defect tests for:

  • Functions or methods.

  • Complete object classes.

Object Class Testing

• Full coverage requires testing all operations, attributes, and states of an object class.

• Complex due to inheritance and object interactions.

Automated Testing

Features

• Automating unit tests with frameworks like JUnit enhances efficiency.

• Helps track code changes and maintain test coverage.

Choosing Unit Test Cases

• Tests must reflect both normal and abnormal use cases.

• Focus on inputs that are expected to succeed and those that could expose defects.

Testing Strategies

Key Techniques

• Partition testing groups similar inputs into equivalence partitions for testing.

• Use guideline-based testing from prior error experiences to choose test cases.

Component Testing

• Tests composed of multiple interacting components to validate interface behaviors.

Interface Testing

Objectives

• Catch faults from interface errors or misunderstandings.

• Test parameter, shared memory, procedural, and message passing interfaces.

Potential Interface Errors

• Common issues include misuse, misunderstanding, and timing errors.

System Testing

Focus Areas

• System testing integrates various components to test interactions and data transfers.

• Checks the emergent behavior of the system as a whole rather than individual units.

User Testing

Importance

• Direct input from users is essential in validating reliability and usability not replicable in controlled tests.

Types

• Alpha Testing: Conducted at the development site.

• Beta Testing: Software released for user experimentation.

• Acceptance Testing: Customers validate readiness for use in their environment.

Test-Driven Development (TDD)

Process Overview

• Write tests prior to developing code, following an incremental development approach.

• Ensures each piece of functionality is continuously validated against testing criteria.

Benefits

• Guarantees code coverage and simplifies the debugging process.

• Provides documentation through tests that state intended functionality.

Key Testing Principles

• Acknowledge that testing can confirm errors exist but not that no errors remain.

• Automation and consistent re-testing are crucial for effective software development and validation.