V&V Notes

Verification and Validation

Objectives

• Introduce software verification and validation.
• Discuss the distinction between verification and validation.
• Describe the program inspection process and its role in V&V.
• Explain static analysis as a verification technique.
• Describe the Cleanroom software development process.

Key Concepts

• Verification: "Are we building the product right?" Conformance to specification.
• Validation: "Are we building the right product?" Meeting user requirements.

V&V Process

• A whole life-cycle process applied at each stage.
• Objectives:
  • Defect discovery.
  • Assessment of system usefulness in operational situations.

V&V Goals

• Establish confidence that the software is fit for purpose.
• Does NOT mean completely free of defects.
• Good enough for its intended use.

Static vs. Dynamic Verification

• Static Verification: Analysis of the static system representation (e.g., software inspections).
• Dynamic Verification: Exercising and observing product behavior (e.g., software testing).

Program Testing

• Reveals the presence of errors, NOT their absence.
• Validation technique for non-functional requirements.
• Should be used with static verification for full V&V coverage.

Types of Testing

• Defect Testing: Discover system defects.
• Validation Testing: Show that the software meets its requirements.

Testing and Debugging

• Distinct processes.
• V&V establishes defect existence.
• Debugging locates and repairs errors.

V&V Planning

• Careful planning is required to optimize testing and inspection.
• Planning should start early.
• Identify the balance between static verification and testing.

Software Inspection

• People examine source representation to discover anomalies and defects.
• Does not require system execution.
• Applied to any system representation.

Program Inspections

• Formalized approach to document reviews for defect detection.
• Defects include logical errors, anomalies, or non-compliance with standards.

Inspection Pre-conditions

• Precise specification available.
• Team members familiar with standards.
• Syntactically correct code.
• Error checklist prepared.

Inspection Roles

• Author/Owner: Fixes defects.
• Inspector: Finds errors.
• Reader: Presents the code.
• Scribe: Records results.
• Moderator: Manages the process.

Inspection Checklists

• Used to drive the inspection.
• Programming language dependent.
• Examples: Initialization, constant naming, loop termination, array bounds, etc.

Inspection Rate

• Expensive process.

Automated Static Analysis

• Software tools for source text processing.
• Discover potentially erroneous conditions.
• A supplement to inspections.

Static Analysis Checks

• Data faults, control faults, input/output faults, interface faults, storage management faults.

Stages of Static Analysis

• Control flow analysis.
• Data use analysis.
• Interface analysis.
• Information flow analysis.
• Path analysis.

Use of Static Analysis

• Valuable for languages with weak typing (e.g., C).
• Less cost-effective for languages with strong type checking (e.g., Java).

Verification and Formal Methods

• Used when a mathematical specification is produced.
• Detailed mathematical analysis to show program conformance.

Arguments for Formal Methods

• Uncover errors by requiring a detailed analysis of requirements.
• Detect implementation errors before testing.

Arguments Against Formal Methods

• Require specialized notations.
• Very expensive.

Cleanroom Software Development

• Defect avoidance rather than defect removal.
• Based on:
  • Incremental development.
  • Formal specification.
  • Static verification using correctness arguments.
  • Statistical testing.

Cleanroom Process Characteristics

• Formal specification using a state transition model.
• Incremental development.
• Structured programming.
• Static verification using rigorous inspections.
• Statistical testing.

Cleanroom Process Teams

• Specification team.
• Development team.
• Certification team.