Software Testing Foundation: Based on ISTQB Standards (Chapter 1: Fundamentals of Testing)

ISTQB

International Software Testing Qualifications Board

November 2002

About ISTQB

Founded, ____________________

Belgium

About ISTQB

Not-for-profit association legally registered in ____________________.

ISTQB® Certified Tester

About ISTQB

Has defined the "__________________________" scheme that has become the worldwide leader in the certification of competences in software testing.

Causes of Software Failures

-Time Pressure

-Complex Codes

-Complex Infrastructure

-Changing Technologies

-Many System Interactions

Error

Mistake

Defect

Fault, bug

Defect

A problem in the program code (or related work product)

Failure

Defects + Environmental Conditions

Failure

System failing to do what it should do or doing something that it should not do

risk

Role of Testing in Software Development, Maintenance and Operations

Benefits of rigorous testing of systems and documentation

Helps reduce the ___________ of problems occurring in delivered systems

defects

Role of Testing in Software Development, Maintenance and Operations

Benefits of rigorous testing of systems and documentation

Contributes to the quality of the software systems if ___________________ are corrected before the system is released

Industry-specific standards

Role of Testing in Software Development, Maintenance and Operations

Benefits of rigorous testing of systems and documentation

May be required to meet

○ Contractual requirements ○ Legal requirements

○ _________________________________

35% & 80%

Role of Testing in Software Development, Maintenance and Operations

Benefits of rigorous testing of systems and documentation

On average, testing user ______________ of the IT budget, but can use ___________ on safety critical systems

functional and non-functional

Testing and Quality

Testing can help:

Measure the quality of the software in terms of defects found, for both _____________________ and _____________________ software requirements and characteristics

confidence & Few defects

Testing and Quality

Testing can help:

Can give ______________in the quality of the software if it finds few or no defects

○ __________________ = confidence in the quality increases

○ Reduces overall level of risk when a properly designed test have passed

defects

Testing and Quality

Testing can help:

When testing finds _______________, the quality of the software increases when those defects are fixed.

Testing

Testing and Quality

Testing can help:

__________________ should be integrated as one of the quality assurance activities

confidence

Testing and Quality

Quality Assurance

Part of quality management focused on providing ________________ that quality requirements will be fulfilled. (ISTQB definition)

root causes of defects

Testing and Quality

Quality Assurance

Lessons should be learned from previous projects. Understanding the ______________________ found in other project, processes can be improved, which should PREVENT those defects from reoccurring, therefore improve the quality of future systems.

Testing

Testing and Quality

Quality Assurance

_________________ is one of the Quality Assurance Activities

Level of Risk

How Much Testing is Enough?

Deciding how much testing is enough should take account the level of risk and project constraints

- Technical

- Safety

- Business

Project Constraints

How Much Testing is Enough?

Deciding how much testing is enough should take account the level of risk and project constraints

- Time

- Budget

- People

sufficient information

How Much Testing is Enough?

Deciding how much testing is enough should take account the level of risk and project constraints

Testing should provide _______________________ to stakeholders to make informed decisions about the release of the software or system being tested, for the next deployment step or handover to customers

Reviews

Test Objectives across the Life Cycle

Of documents (e.g. requirements) and the identification and resolution of issues helps to prevent the defects appearing in the code

Acceptance Testing

Test Objectives across the Life Cycle

To confirm that the system works as expected, and to gain confidence that it has met the requirements

Maintenance Testing

Test Objectives across the Life Cycle

Includes testing that no new defects have been introduced during the implementation of the changes

Quality Evaluation

Test Objectives across the Life Cycle

To assess the quality of the software - to give

information to stakeholders of the risks of

releasing the system at a given time

Operational Testing

Test Objectives across the Life Cycle

To assess ongoing system characteristics such as reliability or availability

Development Testing

Test Objectives across the Life Cycle

To cause as many failures as possible so that the defects in the software are identified and can be fixed

Dynamic Testing

Tester

___________________ reveals failures

Re-testing

Tester

____________________ ensures that the fix really resolves the failure

Debugging

Developer

_________________ locates, analyses and removes the cause of the failure (defects)

test execution

Fundamental Test Process

The most visible part of testing is ___________________. But to be effective and efficient, test plans should also include time to be spent on planning the tests, designing test cases, preparing for execution and evaluating results.

scope and risks

Test Planning and Control

Determining the ______________ and ______________ and identifying the objectives of testing

Defining

Test Planning and Control

________________ the overall approach of testing, including the definition of the test levels and entry and exit criteria

testing activities

Test Planning and Control

Integrating and coordinating the _________________ into the software life cycle activities (acquisition, supply, development, operation and maintenance)

decisions

Test Planning and Control

Making ________________ about what to test, what roles will perform the test activities, how the test activities should be done, and how the test results will be evaluated

Scheduling

Test Planning and Control

_____________________ test analysis and design activities, test implementation, execution and evaluation

resources

Test Planning and Control

Assigning ________________ for the different activities defined

amount

Test Planning and Control

Defining the______________, level of detail, structure and templates for the test documentation

metrics

Test Planning and Control

Selecting ________________ for monitoring and controlling test preparation and execution, defect resolution and risk issues

reproducible test

Test Planning and Control

Setting the level of detail for test procedures in order to provide enough information to support ________________________ preparation and execution

test monitoring

Test Planning and Control

Making decisions based on information from _______________________

Re-prioritizing

Test Planning and Control

_______________________ tests when an identified risk occurs (e.g. software delivered late)

test environment

Test Planning and Control

Changing the test schedule due to availability or unavailability of a _______________________

entry criterion

Test Planning and Control

Setting an ____________________ requiring fixes to have been re-tested (confirmation tested) by a developer before accepting them into a build

test basis

Test Analysis and Design

Reviewing the ___________________ (such as requirements, software integrity level (risk level), risk analysis reports, architecture, design, interface specifications)

testability

Test Analysis and Design

Evaluating ___________________ of the test basis and test objects

test conditions

Test Analysis and Design

Identifying and prioritizing _____________________ based on analysis of test items, the specification, behavior and structure of the software

high level

Test Analysis and Design

Designing and prioritizing ____________________ test cases

test data

Test Analysis and Design

Identifying necessary ___________________ to support the test conditions and test cases

test environment

Test Analysis and Design

Designing the _____________________ setup and identifying any required infrastructure and tools

bi-directional traceability

Test Analysis and Design

Creating _____________________ between test basis and test cases

test cases

Test Implementation and Execution

Finalizing, implementing and prioritizing ) ________________ (including the identification of test data)

test procedures

Test Implementation and Execution

Developing and prioritizing ___________________, creating test data and, optionally, preparing test harnesses and writing automated test scripts

test suites

Test Implementation and Execution

Creating ________________ from the test procedures for efficient test execution

test environment

Test Implementation and Execution

Verifying that the ____________________ has been set up correctly

test basis and test cases

Test Implementation and Execution

Verifying and updating bi-directional traceability between the _____________________ and _____________________

test procedures

Test Implementation and Execution

Executing _____________________ either manually or by using test execution tools, according to the planned sequence

test execution

Test Implementation and Execution

Logging the outcome of _____________________ and recording the identities and versions of the software under test, test tools and testware

expected results

Test Implementation and Execution

Comparing actual results with ______________________

discrepancies

Test Implementation and Execution

Reporting ________________ as incidents and analyzing them in order to establish their cause (e.g. a defect in the code, in specified test data, in the test document, or a mistake in the way the test was executed)

test activities

Test Implementation and Execution

Repeating ____________________ as a result of action taken for each discrepancy, for example, re-execution of a test that previously failed in order to confirm a fix (confirmation testing), execution of a corrected test and/or execution of tests in order to ensure that defects have not been introduced in unchanged areas of the software or that defect fixing did not uncover other defects (regression testing)

test logs

Evaluation Exit Criteria and Reporting

Checking ________________ against the exit criteria specified in test planning

exit criteria

Evaluation Exit Criteria and Reporting

Assessing if more tests are needed or if the __________________ specified should be changed

test summary

Evaluation Exit Criteria and Reporting

Writing a _________________ report for stakeholders

planned deliverables

Test Closure Activities

Checking which ____________________ have been delivered

incident reports

Test Closure Activities

Closing __________________ or raising change records for any that remain open

acceptance

Test Closure Activities

Documenting the __________________ of the system

archiving testware

Test Closure Activities

Finalizing and ____________________, the test environment and the test infrastructure for later reuse

maintenance organization

Test Closure Activities

Handing over the testware to the _______________________

changes needed

Test Closure Activities

Analyzing lessons learned to determine ________________________ for future releases and projects

test maturity

Test Closure Activities

Using the information gathered to improve ______________________

People are driven by objectives set by management and other stakeholders

Psychological Factors Influencing Testing

E.g. "find defects" and "confirm software works"

Testing is often seen as a destructive activity as it identifies failures

Psychological Factors Influencing Testing

However, to find failures requires curiosity, professional pessimism, a critical eye, attention to detail, good communication with development peers, and experience on which to base the error guessing

Communication of defects in a constructive way

Psychological Factors Influencing Testing

Bad feelings between the tester and analysts, designers and developers can be avoided

Testers (and Reviewers) have a different mindset from developers

Psychological Factors Influencing Testing

○ A certain degree of independence (avoiding author bias) often makes the tester more effective at finding defects and failures ○ They can focus in finding Defects

○ They provide an independent view as trained, professional testing resources

Independence makes testers more effective

Psychological Factors Influencing Testing

however developers' familiarity with the software is also useful - they can efficiently find many defects in their own code

PUBLIC

Code of Ethics

Certified software testers shall act consistently with the public interest

CLIENT AND EMPLOYER

Code of Ethics

Certified software testers shall act in a manner that is in the best interests of their client and employer, consistent with the public interest

PRODUCT

Code of Ethics

Certified software testers shall ensure that the deliverables they provide (on the products and systems they test) meet the highest professional standards possible

JUDGMENT

Code of Ethics

Certified software testers shall maintain integrity and independence in their professional judgment

MANAGEMENT

Code of Ethics

Certified software test managers and leaders shall subscribe to and promote an ethical approach to the management of software testing

PROFESSION

Code of Ethics

Certified software testers shall advance the integrity and reputation of the profession consistent with the public interest

COLLEAGUES

Code of Ethics

Certified software testers shall be fair to and supportive of their colleagues, and promote cooperation with software developers

SELF

Code of Ethics

Certified software testers shall participate in lifelong learning regarding the practice of their profession and shall promote an ethical approach to the practice of the profession