QA Fundamentals Glossary

A/B testing

A statistical testing approach to determine which of two components or systems performs better.

ad hoc testing

Informal testing performed without test analysis and test design.

adversarial example

An input to an ML model created by applying small perturbations to a working example that results in the model outputting an incorrect result with high confidence.

adversarial testing

A test technique based on the attempted creation and execution of adversarial examples to identify defects in an ML model.

Agile test leader (Test Manager)

A leadership role that serves agile teams, championing testing and quality at the organizational level.

Agile test team leader

A role that is responsible for maintaining solution quality within an agile delivery team.

application programming interface (API)

A type of interface in which the components or systems involved exchange information in a defined formal structure.

built-in quality

A set of practices to ensure that each solution meets quality standards throughout each increment of development, focusing on constructive quality assurance as a shared responsibility.

causal loop diagram

A graphical representation used to visualize cause-effect relationships and feedback loops in a system.

follow-up test case

A test case generated by applying a metamorphic relation to a source test case during metamorphic testing.

input data testing

A test level that focuses on the quality of the data used for training and prediction by ML models.

metamorphic relation (MR)

A description of how a change in the test inputs from the source test case to the follow up test case affects a change in the expected outputs from the source test case to the follow up test case.

metamorphic testing (MT)

A test technique in which the inputs and expected results are extrapolated from a passing test case using a metamorphic relation.

ML functional performance

The degree to which an ML model meets ML functional performance criteria

ML functional performance criteria

Criteria based on ML functional performance metrics used as a basis for model evaluation, tuning and testing

ML functional performance metrics

A set of measures that relate to the functional correctness of an ML system

ML model

An implementation of machine learning (ML) that generates a prediction, classification or recommendation based on input data

ML model testing

A test level that focuses on the ability of an ML model to meet required ML functional performance criteria and non-functional criteria.

neuron coverage

The coverage of activated neurons in the neural network for a set of tests

pseudo-oracle

An independently derived variant of the test item used to generate results which are compared with the results of the original test item based on the same test.

used to compare its results with those from the original test item, ensuring consistent and relevant evaluation.

quality assistance

An approach to quality management that focuses on a quality culture throughout an organization.

quality coaching

The activities focused on helping an agile organization identify, understand, and deal with quality management, business value, flow, and customer collaboration.

quality culture

An organizational value system that results in an environment to establish and continually improve quality.

sign change coverage

The coverage of neurons activated with both positive and negative activation values in a neural network for a set of tests

sign-sign coverage

The coverage achieved if by changing the sign of each neuron it can be shown to individually cause one neuron in the next layer to change sign while all other neurons in the next layer do not change sign for a set of tests.

source test case

A test case that passed and is used as the basis of follow-up test cases in metamorphic testing

test step

A single interaction between an actor and the test object consisting of an input, an action, and an expected result.

threshold coverage

The coverage of neurons exceeding a threshold activation value in a neural network for a set of tests.

value change coverage

The coverage of neurons activated where their activation values differ by more than a change amount in the neural network for a set of tests

virtual test environment

A test environment in which one or more parts are digitally simulated. ISO/IEC 29119-11

visual testing

Testing that uses image recognition to interact with GUI objects. Location visual testing

retrospective

A regular event in which team members discuss results, review their practices, and identify ways to improve.

European Foundation for Quality Management (EFQM) model

A management framework that supports organisations in managing change and improving performance.

transcendent quality

A view of quality based on the perception and feeling of individuals.

automotive SPICE

A process reference model and an associated process assessment model in the automotive industry.

certification

The process of confirming that a component, system, or person complies with specified requirements.

coverage criteria

The criteria to define the coverage items required to reach a test objective.

entry criteria

The set of conditions for officially starting a defined task.

manufacturing based quality

A view of quality measured by the degree that a product or service conforms to its intended design and requirements based on the process used.

how well a product meets its design specifications and requirements during the manufacturing process. It emphasizes conformity to design criteria, which is essential for determining the quality from a manufacturing perspective.

moderator

(1) The person responsible for running review meetings. (2) The person who performs a usability test session.

PRISMA

A systematic approach to risk-based testing that creates a product risk matrix.

product-based quality

A view of quality measured by the degree that well-defined quality characteristics are met. [After Garvin]

quality control

Activities designed to evaluate the quality of a component or system.

root cause analysis

An analysis technique aimed at identifying the root causes of defects.

test level

A specific instantiation of a test process. (No reference to ISO 29119-1)

test process group (TPG)

A collection of specialists who facilitate the definition, maintenance, and improvement of the test processes used by an organization.

test process improvement manifesto

A statement that echoes the Agile manifesto and defines values for improving the test process. Ref: Veenendaal08

Total Quality Management

An organization-wide management approach to quality based on employee participation to achieve long-term success through customer satisfaction.

user-based quality

A view of quality measured by the degree that the needs, wants, and desires of a user are met. [After Garvin]

value-based quality

A view of quality measured by the ratio of the cost to the value received from a product or service. [After Garvin]

V-model

A sequential software development lifecycle model describing a one-for-one relationship between major phases of software development from business requirements specification to delivery, and corresponding test levels from acceptance testing to component t