Quality Assurance II

What is White Box Testing?

Also called clear box or glass box testing. Testers must understand internal structure and logic of the software.

What are White Box Testing Techniques?

1. Code Coverage: Ensure every method, statement, or instruction is executed at least once.

2. Logic Path / Decision Point Coverage:└ Branch coverage - all if/else paths taken

└ Path coverage - all execution paths are tested

3. Mutation Testing: Introduce small code changes (mutants) and ensure tests detect them (kill the mutants).

What are Coverage Methods?

- Function Coverage: Has each function been called?

- Statement Coverage: Has each statement been executed?

- Decision Coverage: Have all decision outcomes been executed?

- Condition Coverage: Have all Boolean expressions evaluated both true and false?

- Branch Coverage: Each branch executed at least once

- Loop Coverage: Has each loop executed 0, 1, >1 times?

- Path Coverage: Has every possible route through code been executed?

Statement Coverage Example?

Decision Coverage Example?

Branch Coverage

What is Mutation Testing?

- Run test suite, confirm software works

- Generate mutants by altering code

- Run test suite against mutants

- If output differs, mutant is killed (test caught issue)

If output same, test suite is inadequate

Regression Tests

Ensure no new bugs after changes- Often automated; require proper versioning

What is Integration Testing?

The entire system is viewed as a collection of subsystems (sets of classes) determined during the system and object design.

Goal: Test all interfaces between subsystems and the interaction of subsystems

Bottom-up Integration

• The subsystems in the lowest layer of the hierarchy are tested individually first

• Then the next subsystems are tested that call the previously tested subsystems

• This is repeated until all subsystems are included

• Drivers are needed

Top-down Integration

• Test the top layer or the high-level subsystem first

• Then, combine all the subsystems that are called by the tested subsystems and test the resulting collection of subsystems

• Do this until all subsystems are incorporated into the test

• Stubs are needed to do the testing.

What is System Testing?

Test complete system against requirements. Includes both functional and non-functional requirements (e.g., GUI, stress, compatibility)

Smoke vs. Sanity Testing?

- Smoke Test: Basic build checks

Sanity Test: Narrow, focused testing on new changes

Acceptance Testing Types?

• Goal: Demonstrate system is ready for operational use

• Written, conducted and evaluated by the customers

- Alpha: Client tests in developer's environment with dev support

- Beta: Client tests in real environment without dev present

What is Non-Functional Testing?

Testing qualities beyond features, such as performance and scalability

Types of Non-Functional Testing?

- Stress Testing: to determine the breakpoint (both data and users)

- Load Testing: Measure performance based on a large number of users

- Volume Testing: Test what happens if large amounts of data are handled

- Configuration Testing: Test the system with each one of the supported software and hardware configurations

- Compatibility Testing: Evaluate the application's compatibility within different environments

Test Design Stages?

1. Test Strategy

2. Test Planning

3. Test Case Design

4. Test Procedure

What is a Test Strategy?

a statement of the overall approach to testing for a software development organization.

Specifies the levels of testing to be done as well as the methods, techniques and tools to be used

What is a Test Plan?

specifies in detail how the test strategy will be carried out for the project.

Specifies items to be tested, levels, order, environment.

What is Test Case Design?

A set of test inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.

Test cases are specified separately at each level: unit, integration, system and acceptance

Test Case Anatomy?

ID, title, related requirement, module, priority, steps, data, expected/actual result, pass/fail status, author, last run, bug ID

Example Test Cases?

Test Procedure

final stage of test design is the test procedure, which specifies the process for conducting test cases

includes the use of

• test harnesses (programs written solely to exercise the software or parts of it on the test cases),

• test scripts (automated procedures for running sets of test cases), or commercial testing tools

Test Report

Summarizes test outputs. Highlights failures. Should be concise, standardized.

Walkthrough vs Code Review vs Inspection?

- Walkthrough: Informal, 2-3 ppl, verify small changes

- Code Review: Formal, 3-5 ppl, for substantial changes

- Inspection: Most formal, for documents/code, finds defects early

What is Inspection?

Structured review with defined rules. Used in mission-critical systems. Includes planning, overview, preparation, meeting, reporting, rework, and follow-up.

Roles in Inspection?

- Moderator: runs process

- Author: wrote item

- Reviewer: finds defects

- Scribe: records

- Management: should NOT attend

Effective Inspections Tips?

- Use structured checklists to guide what to look for

- Assign clear roles (moderator, scribe, reviewers, author)- Keep meetings focused (max 2 hrs)

- Train all participants- Preparation is key: review material ahead of time

- Focus on defects, not opinions or style

Inspection Metrics

- Track number of defects found

- Time spent on inspection

- Size of artifact reviewed (e.g., lines of code)

- Defect types (logic, syntax, performance)

- Common defect triggers

- Use structured walkthroughs, checklists, and paraphrasing to find issues

Code Review Checklist (General)

- Is logic correct?

- Code follows style conventions?

- No duplicate/redundant code?

- Modular, with reusable components?

- Are names meaningful?-

Proper loop/control structures?

Checklist (Performance, Security)

- Code optimized for performance?

- Inputs validated?

- External systems securely accessed?

- Are there any buffer overflows, SQL injection risks, or race conditions?

Documentation Checklist

- Do comments explain why, not just what?

- Are functions documented properly?

- Describe edge cases or 3rd-party dependencies?

- Incomplete areas are clearly flagged (TODO, FIXME)?

Testing Checklist

1. Variable & Constant Declaration Defects

1.1 Descriptive names with naming conventions

1.2 No confusingly similar names

1.3 All variables initialized

1.4 Localize non-local variables

1.5 Replace literals with constants

1.6 Use constants where needed

2. Method Definition Defects

2.1 Descriptive method names

2.2 Validate parameters

2.3 All return paths return correct values

3. Computation Defects

3.1 Check underflow/overflow

3.2 Use parentheses to avoid ambiguity

4. Control Flow Defects

4.1 Ensure all loops terminate

Code Review Tips

- Be respectful, never take critiques personally

- Make feedback actionable and clear

- Explain reasons: "This failed in past because..."- Share resources (e.g., blog posts)- Keep context clear when recommending changes

Static Program Analysis

- Examines code without execution

- No test cases needed

- Checks code structure, detects violations of best practices and common errors

Static Analysis Tools

- FindBugs/SpotBugs: Detects bug patterns

- JLint: General static checker

- PMD: Flags inefficiency and complexity

- ESC/Java2: Advanced checker for specs

- Linters: Broad tool family used widely

What are Metrics?

- Quantitative measurements of code/artifacts

- Track size, complexity, structural/dependency info

- Help predict maintainability and reliability

Metric Types

- Code Size: SLOC (Source Lines of Code)

- Complexity: Parameters, functions

- Structural: Depth of calls, transactions

- Design Complexity: Modularity, coupling

McCabe’s “Cyclomatic Complexity” Metric

If the control flow graph G of program P has e edges and n nodes, then the cyclomatic complexity v of P is

v (P) = e – n + 2

v (P) is the number of linearly independent paths in G

Example

#nodes: 6

#edges: 7

#decisions : 2

#regions: 2

#Independent Paths / Cyclomatic Complexity: #edges - #nodes + 2 = 3

Or

#decisions + 1 = 3

Or #regions + 1 = 3

So, we have 3 independent paths to tests

Exercise

Functional:

1. Insert card → validate

2. Prompt PIN → validate

3. Select "Withdraw"

4. Enter amount

5. Choose account type → validate

6. Ask for receipt

7. Eject card

8. Dispense cash

9. Print receipt

Non-Functional:

Performance: Handle high user load********

Security: PIN + card auth, protect account info

Testing Practices at Big Tech

- Spectrum of Responsibility: Some use dedicated testers, others test as team

- Spectrum of Importance: Some test pre-release, others post-deployment

- Common Practice: Heavy use of automated testing