Software Testing Foundation - Based on ISTQB Standards - Chapter 2

Software Development Models

Providing guidance on the order in which a project should carry out its major tasks and criteria for transitioning between stages

Software Development Models

Different development life cycle models need different approaches to testing

Software Development Models

Software development models, like test strategies, must be adapted to the context of the project and to the product characteristics

Sequential Models

Waterfall

V - model

W - model

Iterative/Incremental Models

Rapid Application Development (RAD)

Rational Unified Process (RUP)

Agile (e.g. Scrum)

V - Model (Sequential Development Model)

A V-model may have more, fewer or different levels of development and testing depending on the project and the software product

Component Testing

The verification/testing of "Code" part in V-Model

Integration Testing

The verification/testing of "Design" part in V-Model

System Testing

The verification/testing of "Specifications" part in V-Model

Acceptance Testing

The verification/testing of "Requirements" part in V-Model

System Integration Testing

System Testing -------> ___________________

Acceptance Testing

System Integration Testing -----------> _______________

Iterative LC - Regression Testing

Testing (including Regression Test) increases at each iteration. Regression Testing is increasingly important on all iterations after the first one.

Iterative LC - Regression Testing

We can apply the principles from other lifecycles, for example V-model principles in each iteration

COTS-Oriented Life Cycle

For _______________ software, the purchaser does not perform component testing, but could perform integration, system and acceptance testing

Non-Component Development and System Integration & Test

COTS-Oriented Life Cycle

The part of the life cycle where testing is required

testing

Testing within a Life Cycle Model (V-Model)

For every development activity, there is a corresponding _________ activity.

Test Analysis and Design

Testing within a Life Cycle Model (V-Model)

Should begin during corresponding development activity.

Draft Design

Testing within a Life Cycle Model (V-Model)

Testers should participate in reviewing documents as soon as the drafts are available.

Different Test Objectives

Testing within a Life Cycle Model (V-Model)

Each test level has test objectives specific to that level.

Component Testing

Test basis:

● Component requirements

● Detailed design

● Code

Component Testing

Typical test objects:

● Components

● Programs

● Data conversion / migration programs

● Data base modules

Objectives of Component Testing

Component Testing

● Testing functionality

● Testing non-functional characteristics (e.g. memory leaks)

● Testing robustness

● Verifying error tolerance

● Verifying interfaces within a module

● Verifying local data, boundary values

Integration Testing

Test basis:

● Software and system design

● Architecture

● Workflows

● Use cases

Integration Testing

Typical test objects:

● Subsystems

● Database implementation

● Infrastructure

● Interfaces

● System configuration and configuration data

Integration Testing

Assembling tested components into a single system

Integration Testing

Tests interfaces and combinations of interacting components

Component Integration Testing

There may be more than one level of integration testing:

Tests the interactions between software components and is done after component testing

System Integration Testing

There may be more than one level of integration testing:

Tests the interactions between different systems or between hardware and software and may be done after system testing

System Testing

Test basis:

● System and software requirement specification ● Use cases

● Functional specification ● Risk analysis reports

System Testing

Typical test objects:

● System, user and operation manuals

● System configuration and configuration data

System Testing

● Verifying the behavior of a whole system or application

System Testing

● The test environment should correspond to the final production environment as much as possible to minimize the risk of environment specific failures

System Testing

● Investigate both functional and nonfunctional requirement of system and data quality characteristics

System Testing

● Effective System Testing needs to be executed without unit or integration defects

System Testing

● Testers also need to deal with incomplete or undocumented requirements

System Testing

● An independent test team often carries out system testing

Acceptance Testing

Test basis:

● User requirements

● System requirements

● Use Cases

● Business processes

● Risk analysis reports

Acceptance Testing

Typical test objects:

● Business processes on fully integrated system

● Operational and maintenance processes

● User procedures

● Forms

● Reports

● Configuration data

Acceptance Testing

● It is often the responsibility of the customers or users of a system; other stakeholders may be involved as well.

Acceptance Testing

● Goal: To establish confidence in the system, parts of the system or specific non-functional characteristics of the system.

Acceptance Testing

● Finding defects is not the main focus in _________ ___________

Acceptance Testing

● ________________ is not necessarily the final level of testing

○ E.g. a large-scale system integration test may come after the acceptance test for a system.

Acceptance Testing

● ______________ may occur at various times in the life cycle, for example:

○ A COTS software product may be acceptance tested when it is installed or integrated ○ Acceptance testing of the usability of a component may be done during component testing

○ Acceptance testing of a new functional enhancement may come before system testing

User acceptance testing

Forms of Acceptance Testing

fitness for use of the system by business users.

Operational (acceptance) testing

Forms of Acceptance Testing

○ Testing of backup/restore

○ Disaster recovery

○ User management

○ Maintenance tasks

○ Data load and migration task

○ Periodic checks of security vulnerabilities

Contract and regulation acceptance testing

Forms of Acceptance Testing

(In order with the term)

○ ________ _________ ________ is performed against a contract's acceptance criteria for producing custom-developed software.

○ Acceptance criteria should be defined when the parties agree to the contract .

○ ________ _________ __________ is performed against any regulations that must be adhered to, such as government, legal or safety regulations.

Alpha and beta (or field) testing

Forms of Acceptance Testing

(In order with the term)

○ ________ _________ is performed at the developing organization's site but not by the developing team.

○ ________ _________, or field-testing, is performed by customers or potential customers at their own locations.

factory acceptance testing and site acceptance testing

Forms of Acceptance Testing

Organizations may use other terms as well, such as ________ _________ ________ and ________ _________ ________ for systems that are tested before and after being moved to a customer's site.

A function to be performed by the software

Background

A test type is focused on a particular test objective, which could be any of the following:

○ Specified by a process flow model, a state transition model, or a plain language specification

A non-functional quality characteristic, such as reliability or usability

Background

A test type is focused on a particular test objective, which could be any of the following:

○ Specified by a performance model or a usability model

The structure or architecture of the software or system

Background

A test type is focused on a particular test objective, which could be any of the following:

○ Specified by a control flow model or a menu structure model

Change related

Background

A test type is focused on a particular test objective, which could be any of the following:

○ i.e., confirming that defects have been fixed (confirmation testing) and looking for unintended changes (regression testing)

Testing of Function (Functional Testing)

● WHAT the system does?

● __________ _________ are based on functions and features (described in documents or understood by the testers) and their interoperability with specific systems

● May be performed at all test levels

● Specification-based techniques may be used to derive test conditions and test cases from the functionality of the software or system

● Considers the external behavior of the software (black-box testing).

Security testing

Types of Functional Testing

○ Investigates the functions (e.g., a firewall) relating to detection of threats, such as viruses, from malicious outsiders.

Interoperability testing

Types of Functional Testing

○ Evaluates the capability of the software product to interact with one or more specified components or systems.

Non-functional Testing

● __________ _________ includes, but is not limited to, performance testing, load testing, stress testing, usability testing, maintainability testing, reliability testing and portability testing.

● It is the testing of "HOW" the system works. ● May be performed at all test levels

● Describes the tests required to measure characteristics of systems and software that can be quantified on a varying scale, such as response times for performance testing.

● Considers the external behavior of the software and in most cases uses black-box test design techniques to accomplish that.

Structural Testing

● ________ ________ may be performed at all test levels.

● Best used after specification-based techniques, in order to help measure the thoroughness of testing through assessment of coverage of a type of structure.

● Tools can be used to measure the code coverage of elements, such as statements or decisions.

● ________ ________ may be based on the architecture of the system, such as a calling hierarchy.

● ________ ________ approaches can also be applied at system, system integration or acceptance testing levels (e.g., to business models or menu structures).

Black Box Testing

Also known as specification-based testing.

White Box Testing

Also known as structure-based testing. Should follow its opposite to measure test thoroughness. Dependent on tools.

Functional

Functional and Non-Functional Testing

"What"

Non-Functional

Functional and Non-Functional Testing

"How"

Confirmation Testing

Testing Related to Changes : Re-testing and Regression Testing

○ After a defect is detected and fixed, the software should be re-tested to confirm that the original defect has been successfully removed.

○ Debugging (locating and fixing a defect) is a development activity, not a testing activity.

Regression Testing

Testing Related to Changes : Re-testing and Regression Testing

○ The repeated testing of an already tested program, after modification, to discover any defects introduced or uncovered as a result of the change(s).

○ Performed when the software, or its environment, is changed. ○ The extent of __________ _________ is based on the risk of not finding defects in software that was working previously.

Regression Testing

Testing Related to Changes : Re-testing and Regression Testing

● Tests should be repeatable if they are to be used for confirmation testing and to assist _______ __________.

● _______ __________ may be performed at all test levels, and includes functional, non-functional and structural testing.

● _______ _______ suites are run many times and generally evolve slowly, so regression testing is a strong candidate for automation.

Repeatable tests

Testing, Retesting and Regression Testing

are useful for retesting and regression testing.

Automated tests

Testing, Retesting and Regression Testing

are especially useful for regression testing

Maintenance Testing

● __________ ________ is done on an existing operational system, and is triggered by modifications, migration, or retirement of the software or system.

Maintenance Testing

● Modifications include:

○ Planned enhancement changes

○ Corrective and emergency changes

○ Changes to environment (upgrade to OS, db, or COTS)

○ Patches to correct newly-exposed vulnerabilities

● ________ __________ for the retirement of a system may include the testing of data migration or archiving if long data-retention periods are required