Software Testing

is intended to show that a program does what it is intended to do and to discover program defects before it is put into use.

Testing

Can reveal the presence of errors NOT their absence.

Program Testing

is part of a more general verification and validation process, which also includes static validation techniques

Testing

For custom software, this means that there should be at least one test for every requirement in the requirements document.

Program Testing Goals

For generic software products, it means that there should be tests for all of the system features, plus combinations of these features, that will be incorporated in the product release.

Program Testing Goals

To demonstrate to the developer and the customer that the software meets its requirements.

Program Testing Goals

To discover situations in which the behavior of the software is incorrect, undesirable or does not conform to its specification.

Program Testing Goals

To demonstrate to the developer and the system customer that the software meets its requirements

Validation Testing

A successful test shows that the system operates as intended

Validation testing

To discover faults or defects in the software where its behaviour is incorrect or not in conformance with its specification

Defect Testing

A successful test is a test that makes the system perform incorrectly and so exposes a defect in the system.

Defect Testing

is to establish confidence that the system is ‘fit for purpose’.

Verification and Validation

The level of confidence depends on how critical the software is to an organization.

Software purpose

Users may have low expectations of certain kinds of software

User expectation

Getting a product to market early may be more important than finding defects in the program.

Marketing Environment

Concerned with analysis of the static system representation to discover problems

Software inspections

Concerned with exercising and observing product behavior

Software testing

People check the system’s documents or code to find mistakes and defects.

Software Inspections

Proven to be effective in finding program errors.

Software Inspections

are complementary and not opposing verification techniques.

inspections and testing

can check conformance with a specification but not conformance with the customer’s real requirements

Inspections

cannot check non-functional characteristics such as performance, usability, etc

Inspections

where the system is tested during development to discover bugs and defects.

Development Testing

where a separate testing team test a complete version of the system before it is released to users.

Release Testing

where users or potential users of a system test the system in their own environment

User testing

includes all testing activities that are carried out by the team developing the system.

Development testing

where individual program units or object classes are tested; focus on testing the functionality of objects or methods

Unit Testing

where several individual units are integrated to create composite components; focus on testing component interfaces

Component testing

where some or all of the components in a system are integrated and the system is tested as a whole; focus on testing component interactions

System testing

is the process of testing individual components in isolation.

Unit testing

• Individual functions or methods within an object

• Object classes with several attributes and methods

• Composite components with defined interfaces used to access their functionality

Unit testing

Complete test coverage of a class involves

Object class testing

where you initialize the system with the test case, namely the inputs and expected outputs

Setup

where you call the object or method to be tested

call

where you compare the result of the call with the expected result.

assertion

where you identify groups of inputs that have common characteristics and should be processed in the same way.

Partition testing

where you use testing guidelines to choose test cases.

Guideline-based testing

are often composite components that are made up of several interacting objects.

Software components

Data is passed from one function/method to another.

Parameter interfaces

Block of memory is shared between procedures or functions.

Shared memory

One module offers a set of procedures for others to call.

Procedural interfaces

Sub-systems request services from other sub-systems

Message Passing Interface

A calling component calls another component and makes an error in its use of its interface e.g. parameters in the wrong order.

Interface misuse

A calling component embeds assumptions about the behaviour of the called component which are incorrect.

Interface Misunderstanding

The called and the calling component operate at different speeds and out-of-date information is accessed

Timing errors

to identify system interactions can be used as a basis for system testing.

Use case testing

is an approach to program development in which you inter-leave testing and code development

TDD

was introduced as part of agile methods such as Extreme Programming. However, it can also be used in plan-driven development processes.

TDD

is testing the system to check that changes have not ‘broken’ previously working code

Regression testing

is the process of testing a particular release of a system that is intended for use outside of the development team.

Release Testing

is to convince the supplier of the system that it is good enough for use

Release Testing

is usually a black-box testing process where tests are only derived from the system specification.

Release testing

is a form of performance testing where the system is deliberately overloaded to test its failure behaviour

Stress testing

usually involve planning a series of tests where the load is steadily increased until the system performance becomes unacceptable

Performance tests

is a stage in the testing process in which users or customers provide input and advice on system testing.

User or customer testing

Users of the software work with the development team to test the software at the developer’s site

Alpha Testing

A release of the software is made available to users to allow them to experiment and to raise problems that they discover with the system developers.

Beta Testing

Customers test a system to decide whether or not it is ready to be accepted from the system developers and deployed in the customer environment. Primarily for custom systems.

Acceptance Testing