Software Development Lifecycle Testing Concepts

Testing Throughout the Software Development Lifecycle

Keywords

• Acceptance testing, black-box testing, component integration testing, component testing, confirmation testing, functional testing, integration testing, maintenance testing, non-functional testing, regression testing, shift left, system integration testing, system testing, test level, test object, test type, white-box testing

Learning Objectives for Chapter 2:

2.1 Testing in the Context of a Software Development Lifecycle

• FL-2.1.1 (K2) Explain the impact of the chosen software development lifecycle on testing

• FL-2.1.2 (K1) Recall good testing practices that apply to all software development lifecycles

• FL-2.1.3 (K1) Recall the examples of test-first approaches to development

• FL-2.1.4 (K2) Summarize how DevOps might have an impact on testing

• FL-2.1.5 (K2) Explain shift left

• FL-2.1.6 (K2) Explain how retrospectives can be used as a mechanism for process improvement

2.1. Testing in the Context of a Software Development Lifecycle (SDLC)

An SDLC model is a high-level representation of the software development process, specifying how different phases and activities relate logically and chronologically. Various SDLC models include:

• Sequential models (e.g., waterfall, V-model)

• Iterative models (e.g., spiral, prototyping)

• Incremental models (e.g., Unified Process)

Detailed methods and Agile practices can include:

• Acceptance Test Driven Development (ATDD)

• Behavior Driven Development (BDD)

• Domain Driven Design (DDD)

• Extreme Programming (XP)

• Feature Driven Development (FDD)

• Kanban, Lean IT, Scrum, Test Driven Development (TDD)

2.1.1 Impact of the Software Development Lifecycle on Testing

Testing must adapt to the SDLC. The choice of SDLC affects:

• Scope and timing of test activities (test levels and types)

• Level of detail in test documentation

• Selection of test techniques and approaches

• Extent of test automation

• Roles and responsibilities of testers

In sequential models, initial phases involve testers in requirement reviews, test analysis, and test design, with dynamic testing in later phases. Iterative models allow testing, both static and dynamic, across all levels each iteration, necessitating fast feedback and extensive regression testing. Agile assumes ongoing change, favoring lightweight documentation and extensive test automation.

2.1.2 Software Development Lifecycle and Good Testing Practices

Good practices across all SDLCs include:

• Corresponding test activities for every development activity, ensuring quality control

• Different test levels with specific objectives for comprehensive, non-redundant testing

• Early initiation of test analysis and design during development phases

• Involvement of testers in reviewing work products early for defect detection (supporting shift left)

2.1.3 Testing as a Driver for Software Development

Development approaches like TDD, ATDD, and BDD use tests to guide development, embodying principles of early testing and shift left:

• Test-Driven Development (TDD): Tests drive coding instead of extensive design.

• Acceptance Test-Driven Development (ATDD): Tests created from acceptance criteria in system design.

• Behavior-Driven Development (BDD): Tests written in a natural language format for stakeholders using Given/When/Then.

These approaches facilitate maintaining automated tests to ensure code quality during future refactoring.

2.1.4 DevOps and Testing

DevOps aims to synchronize development and operations to meet common goals. This requires cultural shifts to value all functions equally, promoting autonomy, rapid feedback, and integrated practices through continuous integration (CI) and continuous delivery (CD). Benefits include:

• Quick feedback on code quality and effects of changes

• Shift left in testing via high-quality code submissions with tests

• Encouragement of automation via CI/CD, enhancing test environment stability

• Increased visibility of non-functional qualities (e.g., performance)

Challenges include defining a delivery pipeline, maintaining CI/CD tools, and the complexities of test automation.

2.1.5 Shift Left

Shift left advocates for conducting testing earlier in the SDLC. Good practices include:

• Specification reviews highlighting defects

• Writing test cases pre-coding and utilizing test harnesses

• Employing CI/CD for fast feedback

• Conducting static analysis before dynamic testing

• Starting non-functional testing at earlier component levels

Shift left might require additional training and effort upfront but can decrease later costs and efforts when stakeholders buy in.

2.1.6 Retrospectives and Process Improvement

Retrospectives occur at project milestones or releases. Participants discuss successes and failures, emphasizing continuous improvement documented in test completion reports. Benefits for testing include:

• Enhanced test effectiveness/efficiency

• Higher quality of test artifacts

• Strengthened team collaboration

• Improved quality of requirements

2.2 Test Levels and Test Types

Test levels are organized groups of activities at various software development phases, while test types group activities focused on specific quality characteristics.

2.2.1 Test Levels

The five test levels include:

• Component Testing: Focuses on isolated components, often using test frameworks.

• Component Integration Testing: Tests interactions between components, guided by integration strategies.

• System Testing: Assesses overall system capabilities and includes functional and non-functional testing.

• System Integration Testing: Tests system interfaces with external systems, necessitating appropriate environments.

• Acceptance Testing: Validates readiness for deployment, often conducted by end-users.

2.2.2 Test Types

Key test types include:

• Functional Testing: Evaluates what the system should do for completeness and correctness.

• Non-Functional Testing: Assesses how well the system performs using ISO/IEC 25010 categories:

  • Performance efficiency

  • Compatibility

  • Usability

  • Reliability

  • Security

  • Maintainability

  • Portability

  • Safety

• Black-Box Testing: Tests based on external specifications without structural knowledge.

• White-Box Testing: Tests derived from internal structure focusing on code coverage.

2.2.3 Confirmation Testing and Regression Testing

Testing after changes includes:

• Confirmation Testing: Validates defect fixes, potentially involving rerunning all previous tests or simpler confirmations.

• Regression Testing: Ensures no adverse effects from changes, requiring thorough impact analysis and is a candidate for automation.

2.3 Maintenance Testing

Maintenance encompasses corrective, adaptive, and preventive actions with triggers including planned enhancements, upgrades to environments, and retirement activities. The scope of maintenance testing is contingent on the risks and scale of changes. Various triggers include:

• Modifications (planned or hotfixes)

• Environmental upgrades or migrations

• System retirement, necessitating archiving and data retrieval tests.