Chapter 2: Start to 2.1.3 (Testing As A Driver For Software Development)

True

Testing does not exist in isolation; test activities are related to software development activities. (T/F)

Software Development Lifecycle (SDLC)

is an abstract, high-level model representation of the software development process. It defined how different development phases and types of activities performed within this process relate to each other.

Software Development Lifecycle (SDLC)

Examples of them include: Sequential development models (e.g., waterfall model, V-model), iterative development models (e.g., spiral model, prototyping), and incremental development models (e.g., Unified Process).

Testing

The process consisting of all life cycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects.

True

Testing activities must be adapted to the SDLC to succeed. (T/F)

Testing

The choice of the SDLC impacts on the:

- Scope and timing of test activities (e.g., test levels and test types)

- Level of detail of test documentation

- Choice of test techniques and test approach

- Extent of test automation

- Role and responsibilities of a tester

- Skills and Resources required for Testing

Sequential Development Model

Describes the software development process as a linear, sequential flow of activities.

- This means that any phase in the development process should begin when the previous phase.

- In theory, there is no overlap of phases, but in practice, it is beneficial to have early feedback from the following phase.

Requirements gathering and Design

Sequential Development Model

In the initial phases of the sequential development models, like ___________ __________ and _________, tester's typically participate in requirement reviews, conduct test analysis and design, tests which will be executed later once testable code becomes available.

development phase

Sequential Development Model

Since executable code is usually created in the later phases such as the __________ __________, dynamic testing typically cannot be performed early in the SDLC before the Development stage where code is developed.

Waterfall Model

Shows the steps in sequence where the customer requirements are progressively refined to the point where coding can take place. This type of model is often referred to as linear or sequential model.

Waterfall Model

- Each work or product is completed before moving on to the next.

- Testing is carried out once the code has been fully developed.

- Once testing is complete, a decision can be made on whether the product can be released into the live environment.

Additionally, testers are involved after development is done and the testing at the end serves as a quality check. The product can be accepted or rejected at this point.

Requirement Gathering

In the Waterfall Model, this is where:

- Review and Analyze Requirement Documents

- Identify testable requirements

- Create a test plan

System Design

In the Waterfall Model, this is where:

- Analyze System Requirements

- Create test scenarios

- Identify test data and environment

Development

In the Waterfall Model, this is where:

- Developers code and test

Testing

In the Waterfall Model, this is where:

- Test Execution

- Integration Testing

- System Testing

- Acceptance Testing

Waterfall Model

The disadvantages of this model are:

- Finding defects at the end (during testing), could result in substantial costs associated with defect fixing.

- The commercial and financial aspects associated with cost of fixing the defects could be very high - maybe even more than the entire project budget.

- We cannot reject the parts of the system found to be defective and release the rest because the removal of software is often not a clean-cut activity as this could cause the system to become unusable.

Entry Criteria for Testing - Typical Examples:

SCREENSHOT NA LANG IN SLIDE 11

Exit Criteria for Testing - Typical Examples:

SCREENSHOT NA LANG IN SLIDE 12

Test Coverages in Entry/Exit Criteria

- Ensure that the planned test coverage has been achieved.

- Reduces the risk of untested areas which could harbor defects.

Entry/Exit Criteria for Defect Resolution

- Ensures that all critical and high-severity defects are resolved and verified.

- Allows the teams to take control actions where necessary in the event of inadequate outcomes from testing.

- Allows the team to decide if the next development/testing stage/level can begin.

- Residual risks related to unresolved defects are documented and communicated.

Residual Risk

Refers to the remaining risk after all planned measures to mitigate risk have been implemented., it includes any potential potential issued that remain in the system after testing is complete, such as: Defects that are Unresolved or Deferred, Test Conditions or Functionalities or Test Configurations that remain untested (ex: maybe due to resource unavailability or low risk impact that it "Accepted")

Entry Criteria

Residual Risk:

- ________ _________ help teams ensure that testing begins on a solid foundation, which helps to minimize the residual risk for example: allows teams to adequately prepare for coverage and ensure availability of test environment.

Exit Criteria

Residual Risk:

- Similarly, _________ __________ highlights residual risk (defects unresolved) to ensure stakeholders are informed so that they can decide whether to accept the risk as "known issues".

Verification

The V-Model

"Confirmation by examination and through the provision of objective evidence that specified requirements have been fulfilled."

Validation

The V-Model

"Confirmation by examination and through provision of objective evidence that the requirements for a specific intended use or application have been fulfilled."

V-Model

The essence of the ________ is:

- To graphically show the stages of the development life cycle (including the testing stages).

- To graphically show the relationship between each development stage and the related testing stage.

- To demonstrate simply that once a basis for testing has been produced (having been verified) then Planning and Preparation can commence.

- To depict the test stages for which the document is to be a source.

Acceptance Testing

V-Model: Sequence of V-Model

Performed by users; end to end testing.

Business Requirements <------------------> ____________________

System Integration Testing

V-Model: Sequence of V-Model

Performed by testers; dynamic testing; manual or automated.

Functional Specification (FRS) <------------------> ____________________

System Testing

V-Model: Sequence of V-Model

Performed by testers; dynamic testing; manual or automated.

Technical Specification <------------------> ____________________

Component Integration Testing

V-Model: Sequence of V-Model

Typically performed by developers.

Design Specification <------------------> ____________________

Component Testing

V-Model: Sequence of V-Model

Typically performed by developers.

Code <------------------> ____________________

Characteristics of Sequential Model

• They need business needs and requirements to be well defined before development can begin

• It require a development phase to be complete before the next one can begin. (Requirements to be fully defined before Design can begin, Design fully developed before Coding can begin, Tests fully developed before execution can begin)

• Level of (detail) Documentation is based on the complexity of the project, organizational standards, regulatory requirements and specific project requirements and is typically formal.

• In this, in the initial phases testers and other stakeholders can typically participate in requirement reviews, test analysis, and test design.

Characteristics of Sequential Models

• The executable code is usually created in the later phases, so typically dynamic testing cannot be performed early in the SDLC before development of code is complete.

• Customers/Business Representatives often experience the product only during acceptance testing (Tunnel Effect) and are rarely involved during development

• Change management is sequential models is complex and when a change is accepted, it requires changes to be implemented through layers of development documentation. Hence this development methodology may not be suitable for unpredictable or rapidly evolving business needs which require rapid changes to the solution.

Testing in Sequential Models

• Testers create and execute tests on the software under test (verify) and provide reports to the stakeholders.

• Applicable test techniques are used in documenting Test Cases to demonstrate test coverage criteria.

• Testing typically involves manually executing test cases so test cycles tend to be longer.

• Automated Test Scripts are developed from well defined and documented test cases and test scripts.

• Automated tests may run parallelly with manual testing

• Test Automation is often introduced late in the Testing and tend to focus on System and Acceptance tests.

• Emphasis of automation is for Regression Testing and time-consuming tests

Testing in Sequential Models

• Testing in sequential development models need (and assumes ) stable and well defined requirements.

Each phase of development has a corresponding test level

• Tests are divided into categories or test types according to their objectives, which are pursued with differing priorities on different test levels:

• Comprehensive test planning is an important part of the testing process.

• Testers should ideally be involved in validating requirements and design.

• Test documentation is generally detailed (based on formality - including test plans, test cases and test scripts)

Tunnel Effect

The _________ ________ in project management is where the customer has little or no visibility of the product during the course of the development project. Therefore, they only judge at the very end of the project if the deliverable satisfies his need or not.

Iterative Incremental Model

While Sequential Models need requirements to be fully developed and follow a linear progression, ________ ________ ________ are characterized by:

• Adaptability of development for evolving business needs

• Continuous refinement of development process

• Customer involvement and collaboration

• Rapid feedback mechanisms

They are particularly well-suited for projects where requirements are expected to evolve, and there is a need for regular feedback and adaptation. Modern software development practices often favor this for dynamic and evolving project requirements.

Incremental Development

is an approach to software development where a system is designed, implemented, and delivered in small, manageable increments or pieces.

• Each increment could represent a partial implementation of the complete system, adding new features or functionality to the existing project.

• This increment is then reviewed, and feedback from stakeholders is used to make improvements and plan the next iteration.

Incremental Development

• The goal of incremental development is to deliver a working and usable system/workflow gradually by removing bottlenecks, allowing for flexibility, adaptability, and continuous refinement throughout the development process.

Examples:

(Agile) KANBAN (KanBan - Visualizing progress on a whiteboard)

Rational Unified Process

Agile methodologies

follow specific principles and practices outlined in the Agile Manifesto. Some popular examples include: KANBAN, SCRUM, EXTREME PROGRAMMING.

They follow specific principles and practices that promote flexibility, customer collaboration, and continuous improvement and are different them other iterative incremental models like RUP (prescriptive and has definite phases Inception, Elaboration, Construction, Transition) and the Spiral Model (focuses heavily on risk assessment and requires significant documentation and management involvement.)

Agile methodologies

The four core values of _____ software development as stated in the ______ Manifesto are as follows:

1. Individuals and interactions over processes and tools

2. Working software over comprehensive documentation

3. Customer collaboration over contract negotiation

4. Responding to change over following a plan

SCRUM

is a software development approach where the development project is broken down into smaller releases and each release is further broken down into smaller sprints (iterations).

_______ specifics at a glance:

Product Backlog: A prioritized list of features, enhancements, and fixes required for the product.

Sprint Backlog: A subset of the product backlog items selected for implementation in the current sprint (Iteration) Increment: The product backlog items implemented and "Done" during a sprint, a potentially shippable product (MVP)

Sprint: A time-boxed iteration (usually 2-4 weeks) during which a potentially shippable product increment(MVP) is developed.

SCRUM

Each sprint undergoes the development process in multiple iterations in time boxes so that each iteration provides an increment of the product that can be deployed.

• During an iteration, a set of tasks or user stories is selected and worked on, with the goal of producing a "potentially shippable" product increment after the acceptance tests pass (Minimum Viable Product).

• Collaborative Sprint Planning involving the Developers (incl Testers) and the Product Owner allows the team to clarify, refine and plan the implementation of the user stories for the sprint with clear, defined, testable acceptance test criteria.

• The stakeholders that represent the business decide whether to deploy the MVP for use after the acceptance tests pass.

• The process is repeated in further iterations (sprints) until the software meets the desired quality and functionality.

Release planning

In iterative SDLCs, typically two kinds of planning occur:

_________ ________ looks ahead to the release of a product, defines and re-defines the product backlog, and may involve refining larger user stories into a set of smaller user stories. It also serves as the basis for the test approach and test plan across all iterations.

Iteration planning

In iterative SDLCs, typically two kinds of planning occur:

_________ ________ looks ahead to the end of a single iteration and is concerned with the iteration backlog.

Release planning

Testers involved in _______ _______:

• participate in writing testable user stories and acceptance criteria

• participate in project and quality risk analyses

• estimate test effort associated with user stories

• determine the test approach and plan the testing for the release.

Iteration planning

Testers involved in _______ _______:

• participate in the detailed risk analysis of user stories,

• determine the testability of user stories,

• break down user stories into tasks (particularly testing tasks),

• estimate test effort for all testing tasks,

• identify and refine functional and non-functional aspects of the test object.

Rational Unified Process

ITERATIVE INCREMENTAL MODELS - CHARACTERISTICS

Iterative Incremental models of development break down the scope of product development into smaller iterations and increments depending on the project goals or objectives.

EXAMPLES:

• ________ ________ _________: _____ helps managing the complexity of large projects by breaking them into smaller, more manageable iterations. Frequent integration of developed code through the iterations ensures that the system is consistently tested and validated, reducing integration issues.

Spiral model

ITERATIVE INCREMENTAL MODELS - CHARACTERISTICS

Iterative Incremental models of development break down the scope of product development into smaller iterations and increments depending on the project goals or objectives.

EXAMPLES:

• _______ ______: objective is to address high-risk elements early in the project lifecycle through iterative risk analysis and mitigation. Each iteration (or spiral) refines the product further, improving its accuracy and alignment with user needs.

Kanban

ITERATIVE INCREMENTAL MODELS - CHARACTERISTICS

Iterative Incremental models of development break down the scope of product development into smaller iterations and increments depending on the project goals or objectives.

EXAMPLES:

• _________: By focusing on continuous delivery, _______ ensures that small, incremental updates are delivered regularly, enhancing responsiveness to change. The model helps visualize the workflow and identify bottlenecks, improving overall process efficiency (Does not have Time Boxes)

SCRUM

ITERATIVE INCREMENTAL MODELS - CHARACTERISTICS

Iterative Incremental models of development break down the scope of product development into smaller iterations and increments depending on the project goals or objectives.

EXAMPLES:

• __________: Divides the project into time-boxed sprints, enabling incremental delivery of functional product increments. Each sprint ends with a review and retrospective, allowing the team to gather feedback and continuously improve. It also helps overcome uncertainties in requirement evolution and the associated risk by providing enhanced flexibility.

Stakeholder Feedback

TESTING THROUGH ITERATIVE INCREMENTAL MODELS

Iterative and Incremental models place a particular emphasis on regular delivery and feedback from stakeholders to mitigate risks and to manage them effectively. The key characteristics of Testing in these models are:

• Early and Frequent Feedback:

• __________ __________: Regular iterations involve stakeholders, including end-users, in reviewing and testing the increments. This ensures that the product meets user expectations and requirements. Further, the increment may undergo UAT before proceeding to the next iteration.

• Continuous integration: Regular builds & automated tests allow early detection of defects leading to quicker resolution of issues.

Continuous integration

TESTING THROUGH ITERATIVE INCREMENTAL MODELS

Iterative and Incremental models place a particular emphasis on regular delivery and feedback from stakeholders to mitigate risks and to manage them effectively. The key characteristics of Testing in these models are:

• Early and Frequent Feedback:

• Stakeholder Feedback: Regular iterations involve stakeholders, including end-users, in reviewing and testing the increments. This ensures that the product meets user expectations and requirements. Further, the increment may undergo UAT before proceeding to the next iteration.

• _________ _________: Regular builds & automated tests allow early detection of defects leading to quicker resolution of issues.

Risk-Based Testing

TESTING THROUGH ITERATIVE INCREMENTAL MODELS

Iterative and Incremental models place a particular emphasis on regular delivery and feedback from stakeholders to mitigate risks and to manage them effectively. The key characteristics of Testing in these models are:

• _________ ________ : Testing efforts are often prioritized based on risk, focusing on the most critical parts of the system first. This helps in mitigating the most significant risks early in the development process.

Growing Test Suites

TESTING THROUGH ITERATIVE INCREMENTAL MODELS

Iterative and Incremental models place a particular emphasis on regular delivery and feedback from stakeholders to mitigate risks and to manage them effectively. The key characteristics of Testing in these models are:

• _____ ______ ______: Test cases are incrementally added as new features and functionalities are developed. This results in a comprehensive test suite over time. (Test Pyramid - TBD in CH 5)

Regression Testing

TESTING THROUGH ITERATIVE INCREMENTAL MODELS

Iterative and Incremental models place a particular emphasis on regular delivery and feedback from stakeholders to mitigate risks and to manage them effectively. The key characteristics of Testing in these models are:

_______ _______: As new increments are developed, regression testing ensures that existing functionality is not broken..

Test Automation

TESTING THROUGH ITERATIVE INCREMENTAL MODELS

Iterative and Incremental models place a particular emphasis on regular delivery and feedback from stakeholders to mitigate risks and to manage them effectively. The key characteristics of Testing in these models are:

_____ _________: To keep up with the frequent iterations and rapid delivery cycles, _______ ______ is heavily utilized. _______ _______ ensure that testing is efficient and can be repeated with each new build (Vital for Regression Testing)

Collaboration and Communication

TESTING THROUGH ITERATIVE INCREMENTAL MODELS

Iterative and Incremental models place a particular emphasis on regular delivery and feedback from stakeholders to mitigate risks and to manage them effectively. The key characteristics of Testing in these models are:

___________ _____ ____________: Testers work in Cross-Functional Teams with developers, business analysts, and other stakeholders, fostering a collaborative environment where issues are quickly identified and resolved.

Test Documentation:

TESTING THROUGH ITERATIVE INCREMENTAL MODELS

Iterative and Incremental models place a particular emphasis on regular delivery and feedback from stakeholders to mitigate risks and to manage them effectively. The key characteristics of Testing in these models are:

____ __________: Extensive Test Documentation may not be produced but is usually created and updated regularly and incrementally, reflecting the latest state of the system.

Iterative Test Planning

TESTING THROUGH ITERATIVE INCREMENTAL MODELS

Iterative and Incremental models place a particular emphasis on regular delivery and feedback from stakeholders to mitigate risks and to manage them effectively. The key characteristics of Testing in these models are:

• ________ ____ ________: Test plans are developed iteratively and refined at the beginning of each iteration. This allows for adjustments based on the results and feedback from previous iterations and the study of quality metrics being tracked through the development.

Good Testing Practices Through Any SDLC

Whichever development methodology is being used, there are certain good practices that testing should follow:

• For every software development activity, there is a corresponding test activity, so that all development activities are subject to quality control

• Different test levels have specific and different test objectives, which allows for testing to be appropriately comprehensive while avoiding redundancy

• Test analysis and design for a given test level begins during the corresponding development phase of the SDLC, so that testing can adhere to the principle of early testing

• Testers are involved in reviewing work products as soon as drafts of this documentation are available, so that this earlier testing and defect detection can support the shift-left strategy

Testing

TESTING AS A DRIVER FOR SOFTWARE DEVELOPMENT - INTRODUCTION

_______ plays a crucial role in the software development life cycle by ensuring that the software meets its requirements, functions as intended, and is free from critical defects. _______ serves as a driver for various aspects of software development, contributing to the overall quality and reliability of the software product.

Tests are generally manifested as automated tests on the build framework to ensure code quality through future adaptations/iterations.

True

TESTING AS A DRIVER FOR SOFTWARE DEVELOPMENT - INTRODUCTION

Each of these approaches implements the principle of early testing and follows a shift-left approach since the tests are defined and automated before the code is written. (T/F)

Test Driven Development - TDD

TESTING AS A DRIVER FOR SOFTWARE DEVELOPMENT

is a software development approach that directs the coding through test. This is a Test First Approach, meaning, instead of starting with extensive software design, developers begin by writing component test cases for the desired functionality of a component.

These test cases guide the coding process, ensuring that the implementation is directly aligned with the requirements and specifications defined by the tests.

Test Driven Development - TDD

TESTING AS A DRIVER FOR SOFTWARE DEVELOPMENT

Write Tests First, Then Code:

• Write a component test for a program or desired functionality. Initially, this test will fail because the feature hasn't been implemented yet.

• Write the minimum amount of code necessary to make the test pass. This ensures that the code meets the specific requirements of the test.

• Refactor : Refactor the code and tests to improve structure, readability, and performance while ensuring that the tests still pass.

ACCEPTANCE TEST DRIVEN DEVELOPMENT - ATDD

TESTING AS A DRIVER FOR SOFTWARE DEVELOPMENT

• Derives End to End Acceptance Tests from acceptance criteria as part of the system design process

• Tests are written before the part of the application is developed to satisfy the tests

ATDD may be described as: "the practice of expressing a functional requirement as concrete examples or expectations prior to story development."

During story development, a collaborative workflow occurs in which examples are written and then automated, granular automated unit tests are developed and the system code is written and integrated with rest of the running, tested software.

The story is "Done" - deemed ready for exploratory testing and other testing when these scope defining automated checks pass."

ACCEPTANCE TEST DRIVEN DEVELOPMENT - ATDD

TESTING AS A DRIVER FOR SOFTWARE DEVELOPMENT

STEP 1. Derive Acceptance Tests from with testable acceptance criteria as part of the system design process

STEP 2: Express the criteria in concrete examples in Acceptance Test Scenarios

STEP 3. Create an Automated End to End Test which will fail the first time when it is run.

STEP 4: Refactor code (Developers may use TDD) until the Acceptance test passes

BEHAVIOR DRIVEN DEVELOPMENT - BDD

TESTING AS A DRIVER FOR SOFTWARE DEVELOPMENT

• ____ emphasizes the behavior of the system as seen from the perspective of its stakeholders, including users (Ex: HRMS system use by HR Manager, Employee etc) developers, testers, maintenance teams etc.

• Expresses the desired behavior of an application with test cases written in a simple form of natural language (Domain Specific Language), which is easy to understand by stakeholders - the format is: Given/When/Then.

• Test cases are then automatically translated into executable tests

BEHAVIOR DRIVEN DEVELOPMENT - BDD

TESTING AS A DRIVER FOR SOFTWARE DEVELOPMENT

The DSLS convert structured natural language statements into executable tests. The result is a closer relationship to acceptance criteria for a given function and the tests used to validate that functionality. _____ aims to create a shared understanding of how the system should behave using a ubiquitous language that is easy for non-technical stakeholders to understand. ATDD encompasses _____ .

• _____ is a natural extension of TDD testing in general.

• _____ uses Given (some initial context), When(an event occurs), Then (an ensured outcome) format for User Story creation.

___________________ _______________ is an extension of test-driven development and makes use of a simple Domain Specific Language.