Testing (DETAILED) - CCEA - SSD

What is the purpose of testing?

Testing is used to detect errors, bugs and defects by executing software and observing its behaviour

What does testing not show?

it does not show the
absence of errors.

What does testing do?

Testing validates and verifies correctness, consistence and
completeness to ensure the software is fit for purpose and meets expectations

What is the focus of testing?

The modern focus of testing is preventative, ensuring it is completed early to ensure quality and save time and money in the long run.

What are the 5 steps in the testing process?

planning
Designing
Setting up
Executing
Recording and reporting

What needs to be considered in the planning portion of the testing process?

-What needs to be tested
-Whos involved
-When will it take place and how long will it last
-How is it preformed
-What recourses are needed

What is involved in the design portion of the testing process?

Someone will need to design and plan the tests or test cases and predict the results of each test

What is involved in the setting up portion of the testing process?

The testing environment needs to be prepared meaning that hardware and software needs to be set up along with test data and databases being created

What is involved in the execution portion of the testing process?

someone needs to run the tests

What is involved in the recording and reporting portion of the testing process?

the testers need to store the results in a common format and in a specified location as various stakeholders in the project will be interested in the results of these tests.

In the waterfall model, when was testing implemented?

Testing was not implemented until the end of the project.

What was a consequence of testing at the end of the waterfall model?

Errors were not discovered until very late in the development process, leading to increased costs and time for corrections.

What was suggested to improve the testing process in software development?

Testing should take place earlier in the process.

What model was developed to allow earlier testing in the software development process?

The 'V-model'.

What does the 'V-model' show?

Different levels of testing could start as soon as each project phase was complete.

Draw the V-model

Who creates the test plan?

The analyst or another person with a similar role, who understands the
analysis and design of the system

What 6 things need to be considered in a test plan?

Items
Software tools
Approach
Test data
Environment
Schedule

Name the 8 types of testing?

Acceptance
Alpha
Beta
Black box
integration
program
system
white box

What is program testing?

A small item e.g., a class or menu screen is being tested

What is the purpose of program testing?

-Test each component independently to identify errors easily.
-Focus on internal behavior of each class/unit,
-Verify that the code performs all required functions correctly as
specified.
-Reduce hard-to-detect and fix defects early.

How is program testing performed?

When testing a class alone, testers may need to create additional
code (test harness) to simulate missing parts of the system.
The test harness can create instances, set attributes, provide
inputs, and capture outputs.
Use both black box testing and white box testing

when is program testing preformed?

In XP, tests are written before the code, guided by the design
spec (test-driven development).

Who preforms program testing?

-Writing and reviewing unit tests requires programming experience.
-Typically, programmers perform unit tests themselves while
coding.
-Ideally, an independent tester should do this, but it rarely
happens in practice.
-Programmers work in pairs to check each other's code.

What is tested in integration testing?

Several associated, interdependent modules or classes are tested
together in a group or subsystem.

What is the purpose of integration testing?

-To ensure that code units or classes work together as intended and
error-free.
-Focus on the flow of data, messages, and control between classes or modules.
-Verify that the code matches the high-level design specification
or architecture.

How is integration testing carried out?

-Create temporary code to replace missing parts of the system.
-Focus on black box testing to check interfaces and message/data
exchange between modules/classes.
-Less focus on internal code details at this stage.

When is integration testing carried out?

-Test cases are created early from design specifications using sequence and class diagrams

Who carries out integration testing?

Responsibility for integration tests depends on integration level:
High-level integration (system-wide): the system analyst may
design/run tests.
Low-level integration (few classes): the programmer may handle tests.

Why is integration testing important for object orientated languages?

classes often depend on each
other.

What is the best practice for integration testing?

Best practice: perform integration tests incrementally, integrating
classes as they are unit tested, rather than waiting until the end.

What is being tested during system testing?

A whole complete software system (all subsystems are integrated
together) including documentation.

What is the purpose of system testing?

-Check if the software meets all functional requirements
-Ensure it also meets non-functional requirements as detailed in the system requirements specification.
-Determine if the system was developed correctly.

How is system testing preformed?

-Treat the whole system as a black box—do not analyze internal code.
-Functional tests focus on input/output behavior.
-Non-functional tests cover performance, reliability, usability,
security, and recovery.
-Tests should run in an environment that simulates the real-world
production system.
-No extra code is needed since the entire system is complete.

When is system testing preformed?

Test cases are created early as soon as the system requirements
specification is available.

Who preforms system testing?

-Tests are run by team members who did not write the code, e.g the analyst or independent testers.
-Execute tests after integration testing is completed and before
acceptance testing begins.

What is tested during acceptance testing?

A whole, complete system

What is the purpose of acceptance testing?

-Focus on the customer/user perspective: Is the system complete,
accepted, and approved for use?
-Confirm the system meets the customer's needs.
-Check if it is fit for operational use.
-Verify that it meets the contract or Terms of Reference criteria.

How is acceptance testing carried out?

-Acceptance testing is like system testing but designed and run by
users or customers, with developers assisting if needed.
-Conducted in either the developer's simulated environment or the customer's real work environment.
-Uses black box testing—users won't see the source code.
-Users are encouraged to test all functionality, including
unexpected use cases with valid/invalid data.
-Alpha testing
-Beta testing

What is Alpha testing?

-Done by test engineers or selected user groups.
-Conducted in a controlled environment at the developer's site
with access to developers and test tools.
-Test data may be created by the developers.

What is beta testing?

-Done by general users in real-world environments.
-Conducted at the user's site using their own hardware, software, and data.
-Users may download software, with no selection process.

When is acceptance carried out?

Design test cases at the start of the project, based on business
requirements.

Who carries out acceptance testing?

-Design test cases in collaboration with the customer.
-Users execute tests after system testing and before
installation/deployment.
-Different user groups may participate in testing.

What level is white box testing?

Mainly at unit test level using programmers who have knowledge of internal logic.

Who is responsible for white box testing?

Developer/Programmer

What is the purpose of white box testing?

Purpose Analysis of algorithms, logic and internal workings.

What required information is needed for white box testing?

Access to code and algorithms.

How is white box testing carried out?

-White box testing ensures all execution paths in the code are
covered.
Ensure:
-Every statement/path is executed at least once.
-All branches in conditional statements (e.g., IF) are tested.
-Loops are tested: skipped, executed once, and multiple times.

White box testing advantages?

-Can check every line of code.
-Easy to know which input values
should be used to test the application.
-Helps to determine how the code
should be fixed/debugged.
-Can tidy up the code and optimise
it.

White box testing disadvantages?

-Testers need programming
expertise and their time is more
expensive.
-In large systems it is difficult to
test every line of code and a defect may be missed.

What level is black box testing?

-Whole system level (Acceptance and System Testing).
-Subsystem level (Integration Testing).
-Single unit level (Unit Testing).

Who is responsible for black box testing?

Depends on Level: e.g. Developer (Unit Test Level), Analyst (System
Test Level), and User (Acceptance Test Level).

What is the purpose of black box testing?

Input/Output behaviour tested. What is inside the box is not a
concern.

What information is required for back box testing?

Base tests on the functional requirements/Use Cases at System Test level or the detailed design specification at unit testing level.

How is black box testing carried out?

-Testers provide inputs and inspect outputs to check correctness
and appropriate error handling.
-Inputs include both those expected to succeed and fail.
-Exhaustive testing is usually
impractical; selected inputs are used instead.
-A common approach is Equivalence Partitioning:
-Use Boundary Value Analysis to test values at and around the
boundaries where errors often occur:
Lower boundary: 0, 1, 2.
Upper boundary: 11, 12, 13.

What is Exhaustive testing

trying all possible inputs

What is Equivalence Partitioning?

Divide inputs into valid and invalid ranges.
Example: For month input (1-12), choose inputs from valid
(e.g., 4), invalid below range (e.g., -3), and invalid above range
(e.g., 17).

black box testing advantages

-Tester may not need any
programmer knowledge and testing
may be performed by independent testers.
-As it tests the behaviour of the
system it highlights any missing
functionality.
-Test cases can be designed as soon
as the functional specifications
are complete.

black box testing disadvantages

-Wide range of possible inputs;
impractical to do exhaustive
testing.
-Impossible to know if every single
line of code is executed as inside
the 'box' is not visible. If it has not
been executed it is not tested!