Software Testing

Product Operation Factors

1) Correctness

2) Efficiency

3) Usability

4) Integrity

5) Reliability

Product Revision Factors

Maintainability, Flexibility, Testability

Product Transition Factors

Portability, Reusability, Interoperability

POFOD

Probability of failures on demand : failures/number of requests

Availability

uptime / time

ROCOF

Rate of occurence of faillures, failure/time

MTBF

Mean time between failures , time/failure

MTTR

Mean time to recover , downtime/failures

FIRST principle

F - Fast : test should run fast

I - Independent : test should not rely on each other

R - Repeatable : results should be consistent

T - Timely : Written tests at a certain time

BICEP principle

B - Boundaries : Min/Max input

I - Invalid input : Illegal tests

C - Constructors/Destructors : Object life cycle

E - Exception : Correct exception handling

P - Performance : Required elements at a certain time

CORRECT principle

C - Conformance : Matches specifications

O - Ordering : Events happen in the right order

R - Range : Input/Output within a limit

R - References : Points to the right object/data

E - Existence : Required elements exists

C - Cardinality : Correct number of elements

T - Time : Events happen in the required time

Code smells

A symptom of a problem, doesn’t tell us what is wrong

Obscure test

A test that is difficult to understand at glance

Eager test

A test verifies too much functionality in a single test case

Conditional test logic

a test with if/else/loops

Assertion roulette

Too many assertions, hard to determine which one failed

Erratic Test

Tests behave inconsistently, sometimes fails without any change to the code

TDD (test-driven development)

Interactive technique for developing software

testing classes in isolation

TDD Method (Red, Green, Blue)

1) Write a failing test

2) Make the test pass

3) Refactor

TDD Method (Fake it)

1) Write the simplest hard-coded solution to make the test pass

2) Used early to get quick feedback & gradually replace it with variables as needed

→ Helps you get a passing test quickly

Triangular technique

1) Write a second test with a different input

2) Generalise your implementation only after seeing patterns from multiple tests

best used when you’re unsure about correct solution

BDD (Behaviour-Driven Development)

Improves collaboration between developers and testers

Gherkin Language

A domain specific language used in BDD to write test scenarios in text plain

1) Feature : Describe functionality

2) Scenario : Specific test case

3) Given : Precondition

4) When : User action

5) Then : Expected results

6) And/But : Additional steps

What is a mutant ?

Slightly modified version of your original code → created by applying a small change , created to test how well your test cases can detect bugs.

Killed Mutant

Test detected the change (mutation)

Live Mutant

Test did NOT detect the change

Equivalent Mutant

Doesn’t change the behaviour, impossible to be killed