Software Engineering 2: Module 5 and 6

Software Testing

Intended to demonstrate that a program performs as designed and to identify program defects before it is deployed.

Software Testing

When you do this to a software, you execute a program using artificial data.

Results are analyzed for errors, anomalies, and non-functional attributes.

Software Testing

Its limitation is it can reveal the presence of errors, but not guarantee their absence.

Role in Software Development

Part of the broader verification and validation (V&V) process.

Complements static validation techniques (e.g., code reviews, inspections).

Program Testing Goals

To demonstrate to the developer and the customer that the software meets its requirements.

To discover situations in which the behavior of the software is incorrect, undesirable or does not conform to its specification.

Ensure functionality under normal use

The first goal leads to validation testing

Focus on defect detection

The second goal leads to defect testing

Validation Testing

To demonstrate to the developer and the system customer that the software meets its requirements.

A successful test shows that the system operates as intended.

Defect Testing

To discover faults or defects in the software where its behavior is incorrect or not in conformance with its specification.

A successful test is a test that makes the system perform incorrectly and so exposes a defect in the system.

System (The Box)

This is the program or software being tested. In a black-box view, its internal workings are not considered; only its external behavior matters.

Input test data

Data is sent into the system.

This corresponds to the inputs specified by the test cases

Ie (Input Environment/Expected Input)

Represents the source of the input data, often the tester or an external system that provides the necessary inputs.

Output Test Results

Data is produced by the system after processing the input.

This is the actual behavior of the system.

Software Verification

The process of checking that the software meets its stated functional and non-functional requirements.

Software Validation

A more general process that aims to ensure that the software meets the customer’s expectations.

Verification and Validation Confidence

Its main goal is to build confidence that the system is fit for purpose.

Confidence Factors of Verification and Validation Confidence

1. Software Purpose: Critical systems require higher assurance.

2. User Expectations: Some software may be expected to have limited functionality.

3. Marketing Environment: Speed to market may outweigh thorough defect detection.

Inspections

Can check conformance with a specification, but not conformance with the customer's real requirements.

Inspections

Cannot check non-functional characteristics such as performance, usability, etc.

Inspections and testing

Complementary and not opposing verification techniques.

Both should be used during the V & V process.

Software Inspections

A static validation technique where people examine system representations—like requirements, design, or test data.

TESTING Planning

Schedules and allocates resources for all testing activities.

TESTING Planning

Defines the testing process, considering available people and time.

TESTING Planning

Creates a test plan that outlines: what will be tested, predicted testing schedule, how results will be recorded

Test Case

A documented set of conditions, inputs, and expected results used to check whether a software feature works correctly.

Development Testing

Three Stages of Testing.

Discover bugs and defects during development.

Release Testing

Three Stages of Testing.

Ensure a complete version of the system meets the stakeholder requirements before release.

User Testing

Three Stages of Testing.

Its goal is to test the system in the user's own environment to decide if it meets their needs.

Development Testing

Includes all testing activities that are carried out by the team developing the system.

Unit Testing

Where individual program units or object classes are tested.

Unit Testing

The process of testing individual components in isolation.

Unit Testing

Its purpose is to be a defect testing process aimed at finding bugs early.

Object Class Testing

Complete test coverage of a class involves: testing all operations associated with an object, setting and interrogating all object attributes, exercising the object in all possible states.

Inheritance

Makes it more difficult to design object class tests as the information to be tested is not localized.

Partition Testing, Guideline-based Testing

Two Strategies in Choosing Test Cases:

Partition Testing

Where you identify groups of inputs that have common characteristics and should be processed in the same way.

Guideline-based Testing

Where you use testing guidelines to choose test cases. These guidelines reflect previous experience of the kinds of errors that programmers often make when developing components.

Testing Guidelines: Sequences

1. Test software with sequences that have only a single value.

2. Use sequences of different sizes in different tests.

3. Derive tests so that the first, middle, and last elements of the sequence are accessed.

4. Test with sequences of zero length.

Component Testing

Where several individual units are integrated to create composite components.

Component Testing

Focus: Testing composite components, which are made up of several interacting objects.

Component Testing

Access: Functionality is accessed through a defined component interface.

Component Testing

Goal: To demonstrate that the component interface behaves according to its specification.

Component Testing

Prerequisite: Assumes unit tests on the individual objects within the component have already been completed successfully.

Interface testing

To detect faults resulting from.

Parameter Interfaces

Interface Types.

Data is passed directly between methods or procedures.

Shared Memory Interfaces

Interface Types.

A block of memory is directly shared between procedures or functions.

Procedural Interfaces

Interface Types.

A sub-system offers a set of procedures that other sub-systems call.

Message Passing Interfaces

Interface Types.

Sub-systems communicate by sending messages to request services from each other.

Interface Errors

Invalid assumptions about how interfaces work.

Interface Misuse

Common Types of Interface Errors.

The calling component makes a mistake when using the interface of the called component (e.g., passing parameters in the wrong order).

Interface Misunderstanding

Common Types of Interface Errors.

The calling component holds incorrect assumptions about the behavior or functionality of the called component.

Timing Errors

Common Types of Interface Errors.

Occur when the calling and called components operate at different speeds, leading to the use of out-of-date information.

System Testing

Where some or all of the components in a system are integrated and the system is tested as a whole.

System Testing

Focuses on verifying the integrated system by assembling components to create a working version.

System Testing

Its primary goal is to check the interactions between components, ensuring they are compatible, behave correctly, and transfer the appropriate data across their interfaces at the right time.

Basis for Testing

Use Cases for System Testing.

Use cases provide a structured foundation for system testing by identifying system interactions.

Forcing Interactions

Use Cases for System Testing.

Testing each use case naturally involves and forces interactions among several system components.

Documentation

Use Cases for System Testing.

Sequence diagrams that are part of the use case documentation clearly map out the specific components and interactions that are being tested.

Testing Policies

Developed to define the required system test coverage.

Testing Policies

These policies ensure critical areas are tested without checking every possibility.

Menu Access

Examples of common testing policies.

All system functions accessible via menus must be tested.

Function Combinations

Examples of common testing policies.

Combinations of functions (like text formatting) accessed via the same menu must be tested together.

Input Validation

Examples of common testing policies.

Where user input is involved, all functions must be tested with both correct and incorrect input.

Test-Driven Development

An incremental approach where testing and coding are tightly inter-woven.

Test-Driven Development

Its core principle is tests are written before the code.

Test-Driven Development

Its process includes developing code in small increments, writing a new test for each increment.

Test-Driven Development

Its critical driver for moving forward is ensuring the newly written code successfully 'passes' its test.

Identify Increment

TTD: Process Activities.

Determine a small piece of required functionality (an increment) that can be implemented quickly.

Write Failing Test

TTD: Process Activities.

Write and implement an automated test specifically for this new functionality. When run with existing tests, this new test must initially fail because the functionality hasn't been written yet.

Implement Functionality

TTD: Process Activities.

Write the necessary code to make the new test pass.

Re-run All Tests

TTD: Process Activities.

Execute the entire test suite (the new test and all previously implemented tests) to ensure everything works (regression testing).

Next Increment

TTD: Process Activities.

Once all tests run successfully, you proceed to the next small chunk of functionality, repeating the cycle.

Code Coverage

Benefits of Test-Driven Development.

Every code segment that you write has at least one associated test so all code written has at least one test.

Regression Testing

Benefits of Test-Driven Development.

Simplified Debugging

Benefits of Test-Driven Development.

When a test fails, it should be obvious where the problem lies. The newly written code needs to be checked and modified.

System Documentation

Benefits of Test-Driven Development.

The tests themselves are a form of documentation that describe what the code should be doing.

Simplified Debugging

When a test fails, it should be obvious where the problem lies. The newly written code needs to be checked and modified.

System Documentation

The tests themselves are a form of documentation that describe what the code should be doing.

Software Change

A key problem for all organizations is implementing and managing change to their existing software systems.

Software

Needs to be updated because new needs appear, the business environment changes, bugs are found, new hardware is introduced, or the system’s performance and reliability must be improved

Importance of Evolution

1. Organizations have huge investments in their software systems - they are critical business assets.

2. To maintain the value of these assets to the business, they must be changed and updated.

3. Most of a large company’s software budget goes to updating existing systems, not creating new ones.

Evolution

The stage in a software system’s life cycle where it is in operational use and is evolving as new requirements are proposed and implemented in the system.

Servicing

At this stage, the software remains useful but the only changes made are those required to keep it operational i.e. bug fixes and changes to reflect changes in the software’s environment.

No new functionality is added.

Phase-out

The software may still be used but no further changes are made to it.

Evolution Processes

1. Software evolution processes depend on the type of software being maintained, The development processes used, and the skills and experience of the people involved.

2. Proposals for change are the driver for system evolution. It should be linked with components that are affected by the change, thus allowing the cost and impact of the change to be estimated.

3. Change identification and evolution continues throughout the system lifetime.

Change Implementation

Involves designing, implementing, and testing system revisions.

Change Implementation

Its key difference is that developers may first need to understand the existing program, especially if they weren’t the original creators.

Change Implementation

Its program understanding is to learn the system’s structure, how it works, and how proposed changes will affect it.

Urgent Changes

May have to be implemented without going through all stages of the software engineering process.

- Fix serious faults to restore normal operation.

- Address unexpected effects from environment changes (e.g., OS upgrades).

- Respond quickly to urgent business needs (e.g., new competitor product).

Handover Problems

When the system was built using agile methods, but the maintenance team prefers a plan based approach.

When the system was developed with a plan-based approach, but the maintenance team wants to use agile methods.

Legacy Systems

Old systems built with outdated technologies, often dependent on older hardware and processes.

Legacy Systems

They include not just software but the entire socio technical setup—hardware, software, libraries, and business procedures.

Application Software

Legacy System Component.

The application system that provides the business services is usually made up of a number of application programs.

Application Data

Legacy System Component.

These are data that are processed by the application system.

They may be inconsistent, duplicated or held in different databases.

System Hardware

Legacy System Component.

Legacy systems may have been written for hardware that is no longer available.

Support Software

Legacy System Component.

The legacy system may rely on a range of support software, which may be obsolete or unsupported.

Business Processes

Legacy System Component.

These are processes that are used in the business to achieve some business objective.

Business policies and rules

Legacy System Component.

These are definitions of how the business should be carried out and constraints on the business.

Use of the legacy application system may be embedded in these policies and rules.

Socio-technical system

Business process

Application software

Platform and infrastructure software

Hardware

Legacy System Layers

Legacy System Replacement

It is risky for a number of reasons:

• Lack of complete system specification.

• Tight integration of system and business processes.

• Undocumented business rules embedded in the legacy system.

• New software development may be late and/or over budget.