Software Engineering 2: Software Testing

Testing

Intended to demonstrate that a program performs as designed and to identify program defects before it is deployed.

Testing

When you do this to a software, you execute a program using artificial data.

Results are analyzed for errors, anomalies, and non-functional attributes.

Testing

Its limitation is it can reveal the presence of errors, but not guarantee their absence.

Role in Software Development

Part of the broader verification and validation (V&V) process.

Complements static validation techniques (e.g., code reviews, inspections).

Program Testing Goals

To demonstrate to the developer and the customer that the software meets its requirements.

To discover situations in which the behavior of the software is incorrect, undesirable or does not conform to its specification.

Ensure functionality under normal use

The first goal leads to validation testing

Focus on defect detection

The second goal leads to defect testing

Validation Testing

To demonstrate to the developer and the system customer that the software meets its requirements.

A successful test shows that the system operates as intended.

Defect Testing

To discover faults or defects in the software where its behavior is incorrect or not in conformance with its specification.

A successful test is a test that makes the system perform incorrectly and so exposes a defect in the system.

System (The Box)

This is the program or software being tested. In a black-box view, its internal workings are not considered; only its external behavior matters.

Input test data

Data is sent into the system.

This corresponds to the inputs specified by the test cases

Ie (Input Environment/Expected Input)

Represents the source of the input data, often the tester or an external system that provides the necessary inputs.

Output Test Results

Data is produced by the system after processing the input.

This is the actual behavior of the system.

Software Verification

The process of checking that the software meets its stated functional and non-functional requirements.

Software Validation

A more general process that aims to ensure that the software meets the customer’s expectations.

Verification and Validation Confidence

Its main goal is to build confidence that the system is fit for purpose.

Confidence Factors of Verification and Validation Confidence

1. Software Purpose: Critical systems require higher assurance.

2. User Expectations: Some software may be expected to have limited functionality.

3. Marketing Environment: Speed to market may outweigh thorough defect detection.

Inspections

Can check conformance with a specification, but not conformance with the customer's real requirements.

Inspections

Cannot check non-functional characteristics such as performance, usability, etc.

Inspections and testing

Complementary and not opposing verification techniques.

Both should be used during the V & V process.

Software Inspections

A static validation technique where people examine system representations—like requirements, design, or test data.

TESTING Planning

Schedules and allocates resources for all testing activities.

TESTING Planning

Defines the testing process, considering available people and time.

TESTING Planning

Creates a test plan that outlines: what will be tested, predicted testing schedule, how results will be recorded

Test Case

A documented set of conditions, inputs, and expected results used to check whether a software feature works correctly.

Development Testing

Three Stages of Testing.

Discover bugs and defects during development.

Release Testing

Three Stages of Testing.

Ensure a complete version of the system meets the stakeholder requirements before release.

User Testing

Three Stages of Testing.

Its goal is to test the system in the user's own environment to decide if it meets their needs.

Development Testing

Includes all testing activities that are carried out by the team developing the system.

Unit Testing

Where individual program units or object classes are tested.

Unit Testing

The process of testing individual components in isolation.

Unit Testing

Its purpose is to be a defect testing process aimed at finding bugs early.

Object Class Testing

Complete test coverage of a class involves: testing all operations associated with an object, setting and interrogating all object attributes, exercising the object in all possible states.

Inheritance

Makes it more difficult to design object class tests as the information to be tested is not localized.

Partition Testing, Guideline-based Testing

Two Strategies in Choosing Test Cases:

Partition Testing

Where you identify groups of inputs that have common characteristics and should be processed in the same way.

Guideline-based Testing

Where you use testing guidelines to choose test cases. These guidelines reflect previous experience of the kinds of errors that programmers often make when developing components.

Testing Guidelines: Sequences

1. Test software with sequences that have only a single value.

2. Use sequences of different sizes in different tests.

3. Derive tests so that the first, middle, and last elements of the sequence are accessed.

4. Test with sequences of zero length.

Component Testing

Where several individual units are integrated to create composite components.

Component Testing

Focus: Testing composite components, which are made up of several interacting objects.

Component Testing

Access: Functionality is accessed through a defined component interface.

Component Testing

Goal: To demonstrate that the component interface behaves according to its specification.

Component Testing

Prerequisite: Assumes unit tests on the individual objects within the component have already been completed successfully.

Interface testing

To detect faults resulting from.

Parameter Interfaces

Interface Types.

Data is passed directly between methods or procedures.

Shared Memory Interfaces

Interface Types.

A block of memory is directly shared between procedures or functions.

Procedural Interfaces

Interface Types.

A sub-system offers a set of procedures that other sub-systems call.

Message Passing Interfaces

Interface Types.

Sub-systems communicate by sending messages to request services from each other.

Interface Errors

Invalid assumptions about how interfaces work.

Interface Misuse

Common Types of Interface Errors.

The calling component makes a mistake when using the interface of the called component (e.g., passing parameters in the wrong order).

Interface Misunderstanding

Common Types of Interface Errors.

The calling component holds incorrect assumptions about the behavior or functionality of the called component.

Timing Errors

Common Types of Interface Errors.

Occur when the calling and called components operate at different speeds, leading to the use of out-of-date information.

System Testing

Where some or all of the components in a system are integrated and the system is tested as a whole.

System Testing

Focuses on verifying the integrated system by assembling components to create a working version.

System Testing

Its primary goal is to check the interactions between components, ensuring they are compatible, behave correctly, and transfer the appropriate data across their interfaces at the right time.

Basis for Testing

Use Cases for System Testing.

Use cases provide a structured foundation for system testing by identifying system interactions.

Forcing Interactions

Use Cases for System Testing.

Testing each use case naturally involves and forces interactions among several system components.

Documentation

Use Cases for System Testing.

Sequence diagrams that are part of the use case documentation clearly map out the specific components and interactions that are being tested.

Testing Policies

Developed to define the required system test coverage.

Testing Policies

These policies ensure critical areas are tested without checking every possibility.

Menu Access

Examples of common testing policies.

All system functions accessible via menus must be tested.

Function Combinations

Examples of common testing policies.

Combinations of functions (like text formatting) accessed via the same menu must be tested together.

Input Validation

Examples of common testing policies.

Where user input is involved, all functions must be tested with both correct and incorrect input.

Test-Driven Development

An incremental approach where testing and coding are tightly inter-woven.

Test-Driven Development

Its core principle is tests are written before the code.

Test-Driven Development

Its process includes developing code in small increments, writing a new test for each increment.

Test-Driven Development

Its critical driver for moving forward is ensuring the newly written code successfully 'passes' its test.

Identify Increment

TTD: Process Activities.

Determine a small piece of required functionality (an increment) that can be implemented quickly.

Write Failing Test

TTD: Process Activities.

Write and implement an automated test specifically for this new functionality. When run with existing tests, this new test must initially fail because the functionality hasn't been written yet.

Implement Functionality

TTD: Process Activities.

Write the necessary code to make the new test pass.

Re-run All Tests

TTD: Process Activities.

Execute the entire test suite (the new test and all previously implemented tests) to ensure everything works (regression testing).

Next Increment

TTD: Process Activities.

Once all tests run successfully, you proceed to the next small chunk of functionality, repeating the cycle.

Code Coverage

Benefits of Test-Driven Development.

Every code segment that you write has at least one associated test so all code written has at least one test.

Regression Testing

Benefits of Test-Driven Development.

Simplified Debugging

Benefits of Test-Driven Development.

When a test fails, it should be obvious where the problem lies. The newly written code needs to be checked and modified.

System Documentation

Benefits of Test-Driven Development.

The tests themselves are a form of documentation that describe what the code should be doing.

Simplified Debugging

When a test fails, it should be obvious where the problem lies. The newly written code needs to be checked and modified.

System Documentation

The tests themselves are a form of documentation that describe what the code should be doing.