Software Testing

Structure-based Testing (White-box)

• Focus: Writing test cases from given control flows using different test design techniques.
• Objective: Recalling data flow testing as structure-based testing.

Experienced-based Testing

• Focus: Writing test cases based on intuition, experience, and knowledge about common defects.

Choice of Design Techniques

• Objective: Listing the factors that influence the selection of the appropriate test design technique for a particular kind of problem.

White-box Testing

• Definition: Testing based on analysis of internal logic (design, code, etc.). Expected results still come from requirements.
• Synonym: Also known as structural testing.
• Nature: A technique for designing tests, not a level of testing.
• Application: Primarily applies to lower levels of testing (e.g., unit and component).

Verification in White Box Testing

• Internal security holes.
• Broken or poorly structured paths in the coding processes.
• The flow of specific inputs through the code.
• Expected output.
• The functionality of conditional loops.
• Testing of each statement, object, and function on an individual basis.

How White Box Testing is Performed

1. Identifying what has to be tested.
2. Plotting on the flowgraph.
3. Creating test cases for every path.
4. Executing the tests.

Types of White-box Testing

• Static Testing
  • Code Walkthrough
  • Code Inspection
• Dynamic Testing
  • Unit Testing
    • Unit Coverage
    • Statement Coverage
    • Decision Coverage
    • Branch Coverage
    • Path Coverage
    • Condition Coverage
    • Code Complexity

Advantages of White Box Testing

• Test cases can be automated with ease.
• Tests can be done without GUI.
• Testing is more efficient.
• Optimization of code can be done as the hidden errors are visible.

Disadvantages of White Box Testing

• It is comparatively a more complex process.
• It can be expensive.
• The tests are usually not detailed; hence they can have errors.
• An experienced resource person is required for testing as it requires technical knowledge.

White-box Testing Techniques

• Statement Coverage
• Decision Coverage
• Branch Coverage
• Path Coverage
• Condition Coverage
• Multiple Condition Coverage
• Finite State Machine Coverage
• Control flow testing
• Data flow testing

Statement Coverage

• Aim: To traverse all statements at least once; each line of code is tested.
• Flowchart: Every node must be traversed at least once.
• Benefit: Helps in pointing out faulty code since all lines are covered.
• Formula: Statement\ coverage = \frac{number\ of\ executed\ statements}{total\ number\ of\ statements} \times 100

Statement Coverage Details

• Covers:
  • Dead code.
  • Unused statements.
  • Unused branches.
  • Missing statements.
• Usage:
  • To check the quality of the code.
  • To determine the flow of different paths of the program.
  • Check whether the source code expected to perform is valid or not.
  • Tests the software’s internal coding and infrastructure.

Drawback of Statement Coverage

• Cannot check the false condition.
• Requires different input values to check all the conditions.
• More than one test case may be required to cover all the paths with a coverage of 100%.

Example #1

• Code: ```
  1. public int euclid (int m, int n){
  2. int r;
  3. if( n > m ){
  4. r=m;
  5. m=n;
  6. n=r; }
  7. r = m % n;
  8. while ( r != 0 ){
  9. m = n;
  10. n = r;
  11. r = m % n; }
  12. return n;
  13. }
      ```
• Scenario 1: If m=1, n=3
  • Executed statements: 1, 2, 3, 4-6, 7, 8, 9-11, 12, 13
  • Number of executed statements = 10
  • Total number of statements = 13
  • Statement Coverage: 10/13 = 76.92%
• Scenario 2: If m=3, n=1
  • Executed statements: 1, 2, 3, 7, 8, 12, 13
  • Number of executed statements = 7
  • Total number of statements = 13
  • Statement Coverage: 7/13 = 53.8%

Decision Coverage

• Reports the true or false outcomes of each Boolean expression.
• Also known as branch coverage or all-edges coverage or edge testing.
• Formula: Decision\ coverage = \frac{number\ of\ decision\ outcomes\ exercised}{total\ number\ of\ decision\ outcomes} \times 100

Example #3

• Code: ```
  1. public int euclid (int m, int n){
  2. int r;
  3. if( n > m ){
  4. r=m;
  5. m=n;
  6. n=r; }
  7. r = m % n;
  8. while ( r != 0 ){
  9. m = n;
  10. n = r;
  11. r = m % n; }
  12. return n;
  13. }
      ```
• Scenario 1: If m=1, n=3
  • Decision outcomes exercised: 2, 3, 4-6, 8, 12
  • Number of decision outcomes exercised= 4
  • Total number of decision outcomes = 8
  • Decision Coverage: 4/8 = 50%

Example #4

• Code: ```
  1. public int euclid (int m, int n){
  2. int r;
  3. if( n > m ){
  4. r=m;
  5. m=n;
  6. n=r; }
  7. r = m % n;
  8. while ( r != 0 ){
  9. m = n;
  10. n = r;
  11. r = m % n; }
  12. return n;
  13. }
      ```
• Scenario 2: If m=3, n=1
  • Decision outcomes exercised: 2, 3, 7, 8, 12
  • Number of decision outcomes exercised= 2
  • Total number of decision outcomes = 8
  • Decision Coverage: 2/8 = 25%

Branch Coverage

• Test cases are designed so that each branch from all decision points are traversed at least once.
• Flowchart: All edges must be traversed at least once.
• Formula: Branch\ coverage = \frac{number\ of\ executed\ branches}{total\ number\ of\ branches} \times 100

Advantages of Branch Coverage

• Allows you to validate all the branches in the code.
• Helps you to ensure that no branch leads to any abnormality of the program's operation.
• Removes issues which happen because of statement coverage testing.
• Allows you to find those areas which are not tested by other testing methods.
• Allows you to find a quantitative measure of code coverage.
• Ignores branches inside the Boolean expressions.

Example #5

• Code: ```
  1. public int euclid (int m, int n){
  2. int r;
  3. if( n > m ){
  4. r=m;
  5. m=n;
  6. n=r; }
  7. r = m % n;
  8. while ( r != 0 ){
  9. m = n;
  10. n = r;
  11. r = m % n; }
  12. return n;
  13. }
      ```
• Scenario 1: If m=1, n=3
  • Executed branches: 2, 3, 4-6, 7, 8, 9-11, 12
  • Number of executed branches= 3
  • Total number of branches= 5
  • Branch Coverage: 3/5 = 60%

Example #6

• Code: ```
  1. public int euclid (int m, int n){
  2. int r;
  3. if( n > m ){
  4. r=m;
  5. m=n;
  6. n=r; }
  7. r = m % n;
  8. while ( r != 0 ){
  9. m = n;
  10. n = r;
  11. r = m % n; }
  12. return n;
  13. }
      ```
• Scenario 2: If m=3, n=1
  • Executed branches: 2, 3, 7, 8, 12
  • Number of executed branches= 2
  • Total number of branches= 5
  • Branch Coverage: 2/5 = 40%

Path Coverage

• Definition: A structured testing technique for designing test cases with the intention to examine all possible paths of execution at least once.
• Formula: Path\ coverage = \frac{number\ of\ path\ covered}{total\ number\ of\ paths} \times 100

Advantages of Path Coverage

• Helps reducing redundant tests.
• Focus on program logic.
• Test cases will execute every statement in a program at least once.

Example #7

• Code: ```
  1. public int euclid (int m, int n){
  2. int r;
  3. if( n > m ){
  4. r=m;
  5. m=n;
  6. n=r; }
  7. r = m % n;
  8. while ( r != 0 ){
  9. m = n;
  10. n = r;
  11. r = m % n; }
  12. return n;
  13. }
      ```
• Scenario 1: If m=1, n=3; Path: 1, 2, 3, 4-6, 7, 8, 12, 13, 9 -11

Example #8

• Code: ```
  1. public int euclid (int m, int n){
  2. int r;
  3. if( n > m ){
  4. r=m;
  5. m=n;
  6. n=r; }
  7. r = m % n;
  8. while ( r != 0 ){
  9. m = n;
  10. n = r;
  11. r = m % n; }
  12. return n;
  13. }
      ```
• Scenario 2: If m=3, n=1; Path: 1, 2, 3, 7, 8, 12, 13, 9-11

Data Flow Testing

• Definition: A family of test strategies based on selecting paths through the program's control flow to explore sequences of events related to the status of variables or data objects.
• Focus: On the points at which variables receive values and the points at which these values are used.

Data Flow Testing Levels

• Static data flow testing
  • Identify potential defects, commonly known as data flow anomaly.
  • Analyze source code.
  • Do not execute code.
• Dynamic data flow testing
  • Involves actual program execution.
  • Bears similarity with control flow testing.
  • Identify paths to execute them.
  • Paths are identified based on data flow testing criteria.

Advantages of Data Flow Testing

• Helps to pinpoint:
  • A variable that is declared but never used within the program.
  • A variable that is used but never declared.
  • A variable that is defined multiple times before it is used.
  • Deallocating a variable before it is used.

Conditional Coverage

• Definition: Reveals how the variables or subexpressions in the conditional statement are evaluated. Expressions with logical operands are only considered.
• Boolean operations: Expressions may have Boolean operations like AND, OR, XOR, which indicate total possibilities.
• Sensitivity: Offers better sensitivity to the control flow than decision coverage.
• Guarantee: Does not give a guarantee about full decision coverage.

Example

• Expression: IF (x < y) AND (a>b) THEN
• Possible combinations:
  • TT
  • FF
  • TF
  • FT

Experienced-based Testing Techniques

• Exploratory Testing
• Error Guessing

Error Guessing

• Requirement: Experienced and good testers are required to recognize the defects in the component.
• How it works: Experienced testers can find weaknesses of a system, making this approach effective after more formal techniques.
• Assumption: Assumptions and guesses are constructed by experienced testers, saving time.
• Success: Absolutely dependent on the skills and experience of the tester.

Exploratory Testing

• Purpose: Examines or explores the software to find out if it works or not.
• Process: Testers continuously make decisions about what to test next and where to spend time.
• Usefulness: Useful when there is limited time and poor specifications are available.
• Involvement: Testers are involved in minimum planning and maximum test execution.
• Objective: Checks formal test processes (planning, test design, test execution, etc.) to ensure that most serious defects are found.

Dynamic Test: Choice of Test Design Techniques

• Basis: Which testing technique is best for a particular project is based on the number of internal and external factors.

Internal Factors

• The model used in the development of the system
• Tester's knowledge and experience.
• Experience with a similar previous system
• Objectives of the test
• Initial documentation of the content and its style.
• Use of life cycle model

External Factors

• Risk Assessment
  • If the risk is high, then it requires more detailed and formal testing. Commercial risk can be affected by quality issue hence the exploratory testing is suitable.
• The requirements of customer and commitments in the contract.
• The type of system used
  • The type of system i.e embedded, graphical, financial etc. will affect the choice of techniques. For example: The financial application consists of a lot of calculations. In this case, boundary value analysis is beneficial.
• Regulatory requirements
  • Some industries have regulatory standards or guidelines that guide the testing techniques to be used.
• Time and budget of the project
  • The available time always affects the choice of testing techniques. If more time is available then we can select more techniques and if limited time is available then find the most important defects only.