Unit 6: Test Management

Term side up. Use semi-colon or colon to separate terms for enumeration.

Define Test Management.

Test management is the planning, monitoring and documenting of testing activities used for product quality assurance.

How is Test Management related to software development complexity?

As software development has become more complex, so have testing efforts, making test management an important part of the development process.

Differentiate Test Management from Project Management.

Project management may be used to organize the entire software development lifecycle (SDLC) while Test management focuses specifically on the software testing process of the SDLC.

What are the two phases of the Test Management process?

Planning Phase; Execution Phase.

What happens during the Planning Phase of Test Management?

Test managers conduct test organization, which entails structuring the testing team and assigning responsibilities to testers and other team members; test planning, in which they map out test strategy and objectives; test estimation, in which they consider what resources and funding the testing will require; and conduct risk analysis, in which they determine what could go wrong during a testing lifecycle

What happens during the Execution Phase of Test Management?

Test managers supervise and monitor the progress of testing activities; conduct issue management to identify problems arising during testing and instruct team members to address them; and create test reports summarizing and analyzing the test process and test data for review by stakeholders

Define Test Organization.

During this phase, a test manager will organize a skilled testing team and define their individual roles and/or activities in the testing process. An effective test organization ensures that all the goals of testing are clearly defined and that team members are working efficiently toward common goals.

What determines the specialized test positions in a test organization?

These positions are often based on the role and experience level of the tester.

Enumerate the responsibilities of a Test Analyst.

Focuses more on the business needs of the customer than on technical aspects of testing; Performs mainly functional testing but also contributes to user-focused, non-functional testing, such as usability, adaptability, installability, or interoperability testing; Uses black-box test techniques and experience-based testing rather than white-box test techniques; Improves the effectiveness of testing using defect prevention techniques.

Enumerate the responsibilities of a Technical Test Analyst.

Recognize and classify the typical risks associated with the performance, security, reliability, portability and maintainability of software systems; Provide technical elements to the planning, design and execution of tests for mitigating performance, security, reliability, portability and maintainability risks; Select and apply appropriate white-box test techniques to ensure that tests provide an adequate level of confidence, based on design coverage; Effectively participate in reviews with developers and software architects applying knowledge of typical defects in the code and architecture; Improve the quality characteristics of code and architecture by making use of different analysis techniques.

Enumerate the responsibilities of a Test Manager.

Manage testing in various software development projects by applying test management processes established for the project team or test organization; Identify test stakeholders and software development lifecycle models that are relevant in a given context; Organize risk identification and risk assessment sessions within any software development lifecycle and use the results to guide testing to meet the test objectives; Define a project test strategy consistent with the organizational test strategy and project context; Continuously monitor and control testing to achieve project goals; Assess and report test progress to project stakeholder.

Define Test Planning.

Test planning is the process of deciding what to test, how testing will be performed, who will be responsible for it, and when the testing activities will take place. It includes defining testing tasks and milestones to monitor progress. The plan is influenced by the project’s scope, risks, resources, and testing policies and may be updated as the project develops and new changes arise.

What does Test Planning cover under Scope and Risks?

Test planners define the objectives of testing and determine what will be covered. They also identify potential risks that may affect the testing process.

What does the Test Approach determine?

The test approach determines how testing will be carried out, including the test levels to be used and the criteria for starting and completing testing activities.

What does the Test Schedule organize?

The test schedule organizes testing activities throughout the software life cycle, including test analysis, design, implementation, execution, and evaluation.

What resources do test planners identify?

Test planners identify the resources needed for testing, such as testers, tools, test environments, and test data.

What are Documentation and Metrics in Test Planning?

Test planning also involves preparing the necessary test documentation and selecting metrics to monitor testing progress, defects, and risks.

What are Entry Criteria?

Are we ready to start testing? Test environment and tools are ready; Code is available; Test data is prepared.

What are Exit Criteria?

Have we done enough testing to stop? Required test coverage is achieved; Defect levels are acceptable; Reliability and remaining risks are evaluated.

Define the Test Plan Document.

The test plan describes the overall approach to testing based on the system’s requirements and use cases. It outlines the testing activities, responsibilities, resources, schedule, and standards to guide the testing process and keep it clear for the test team and stakeholders.

(7) Test Plan Template Format Selections

Test Plan ID, Test Plan Name, Version, Date, Author, Objective, Scope

Describe the Test Plan ID under the Test Plan Template Format.

Unique identifier for the test plan.

Describe the Test Plan Name under the Test Plan Template Format.

Title or name of the test plan.

Describe the Version under the Test Plan Template Format.

Version number of the test plan.

Describe the Date under the Test Plan Template Format.

Date when the test plan was created or updated.

Describe the Author under the Test Plan Template Format.

Name of the person who created the test plan.

Describe the Objective under the Test Plan Template Format.

Goals and objectives of the testing process.

Describe the Scope under the Test Plan Template Format.

Description of the features, functionalities, or areas to be tested.

Define Test Estimation.

Test estimation is the process of predicting the time, effort, people, and resources needed to carry out testing activities. It may be based on previous project data or expert judgment. Accurate estimation helps set realistic testing plans and use resources effectively.

Define Work Breakdown Structure (WBS) as a test estimation technique.

Breaks testing into smaller tasks and estimates the effort for each task.

Define Three-Point Estimation (PERT) as a test estimation technique.

Uses optimistic, most likely, and pessimistic estimates to calculate a more balanced estimate of the testing effort.

Define Experience-Based Estimation.

Uses knowledge from similar projects and the experience of testers or experts.

Define the Wideband Delphi Technique.

Uses estimates from multiple experts and discussion to reach an agreed estimate.

Define Function Point Analysis (FP).

Estimates effort based on the functionality and complexity of the software.

Define Ad-Hoc Estimation.

Uses personal judgment or quick estimates without a formal method. Useful for small or urgent tasks.

Enumerate the test estimation techniques in the presentation.

Work Breakdown Structure (WBS); Three-Point Estimation (PERT); Experience-Based Estimation; Wideband Delphi Technique; Function Point Analysis (FP); Ad-Hoc Estimation.

Define Test Strategy.

A test strategy is a high-level plan that shows how testing will be performed for a software project. It defines the rules for quality checks and takes the testing team through every development phase.

Differentiate Test Strategy and Test Plan.

TEST STRATEGY: It captures the high-level goals and methods for finding defects before a software package is released. TEST PLAN: It specifies tests in extreme detail with detailed roadmap which is beneficial primarily for the testing team.

Enumerate the components of a Test Strategy.

Testing Scope and Planning; Testing Types; Testing Environment and Tools; Risk Management and Quality Gates; Databases For Tracking and Reporting.

Define Testing Scope and Planning.

The testing scope determines what is to be tested and why. This simplifies prioritization for teams, helping them focus on the significant components of the software.

Enumerate the testing types shown as components of a Test Strategy.

Unit Testing; System Testing; Integration Testing; Acceptance Testing.

Define Testing Environment and Tools.

The testing environment consists of the hardware, software, and network configurations that are required to test.

What is Risk Management and Quality Gates in a Test Strategy?

Risk assessment picks up potential issues that may impact the testing/quality of the software. Quality gates, such as entry and exit criteria, define the parameters for the commencement and conclusion of testing.

What is the purpose of Databases for Tracking and Reporting?

Teams require systems for reporting, monitoring, and fixing software problems. Defect tracking works to record and manage the entire lifecycle of a problem from a user perspective.

Enumerate the best practices for developing test strategies.

Understand the users of the software and develop a profile for each user category; State testing objectives explicitly; Develop a continuous improvement approach for the testing process.

Enumerate the steps in writing a Test Strategy.

Consult test policy to see company-wide QA and business principles; Meet project stakeholders to discuss project’s workflows; Create a test strategy draft; Invite stakeholders for a review and amendments; Share the strategy with the QA team to discuss concerns and potential changes; Adopt the strategy and use it.

Define Risk.

Risk is the possibility that an event, problem, or failure may occur and negatively affect the software, users, or project objectives.

What is the role of Testing in relation to risk?

Testing is an important way of reducing uncertainty and increasing confidence that the software will perform as expected.

Why is risk important in testing?

Software systems can have many features, inputs, users, devices, and possible conditions. Due to this, it is usually impossible to test every possible situation.

Enumerate the resources listed in the Risk and Testing section.

Time; Budget; People; Tools and Equipment; Testing Environments.

Define Product Risk.

Product risks are possible problems related directly to the software or system being developed. These risks affect the quality, behavior, reliability, or performance of the software.

Define Project Risk.

Project risks are possible problems that may affect the development and testing process. These risks may delay testing, limit available resources, or make it difficult for the team to complete testing successfully.

What question identifies Product Risk versus Project Risk?

Product Risk: What could go wrong with the software? Project Risk: What could go wrong while developing or testing the software?

Enumerate the examples of Product Risk.

Missing or incorrect functions – the system does not perform a required feature correctly; Incorrect calculations or outputs – the system produces inaccurate information or results; Poor performance – the system becomes slow or unstable when many users access it; Poor usability – users may find the system difficult or confusing to use; Security vulnerabilities – unauthorized users may gain access to information or functions; Unreliable behavior – the system does not consistently produce the expected result.

Enumerate the common Organizational Project Risks.

Lack of testers; Lack of testing experience or training; Poor communication between team members; Limited available resources.

Enumerate the common Technical Project Risks.

Unclear or incomplete requirements; Frequent changes in system requirements; Complex system design; Poor-quality code or system architecture.

Enumerate the common Testing Environment Project Risks.

Test environment is unavailable; Required software cannot be installed; Test devices or servers are not available.

Enumerate the common External Project Risks.

Failure of third-party software or services; Delays from external suppliers; Problems with external APIs or systems.

(4) Enumerate Common Project Risks

Organizational; Technical; Testing Environment; External.

State the simple difference between Product Risk and Project Risk.

Product Risk — A problem may occur inside the software. Project Risk — A problem may occur while the team is developing or testing the software.

Define the risk response option Mitigate.

Take actions to reduce the likelihood or possible impact of the risk before it causes a serious problem.

Define the risk response option Contingency.

Prepare a backup plan that can be used if the risk actually happens.

Define the risk response option Transfer.

Shift the responsibility for handling the risk to another person, team, or organization that is better equipped to manage it.

Define the risk response option Ignore / Accept.

The team may decide to accept the risk when its possible effect is small or when taking action would require more resources than the risk justifies.

Enumerate the four risk response options.

Mitigate; Contingency; Transfer; Ignore / Accept.

Define Functional Testing.

Checks whether the features of the system work according to the specified requirements.

Define Security Testing.

Checks whether the system protects data and prevents unauthorized access.

Define Performance Testing.

Checks how the system behaves under different workloads and numbers of users.

Define Usability Testing.

Checks whether the system protects data and prevents unauthorized access.

Define Regression Testing.

Checks whether recent updates or changes have affected existing functions.

Define Risk-Based Testing.

Is a testing approach where the amount, priority, and type of testing performed are influenced by the risks associated with different parts of the software.

What is the basic principle of Risk-Based Testing?

More important or serious risk → Greater testing attention

How does risk influence Test Priority? Give the example.

Functions that may cause serious problems if they fail can be tested earlier. Example: Login and payment functions may be tested before minor visual features.

How does risk influence Test Coverage, and what may the team check?

Important functions may require more test cases, inputs, and scenarios. The testing team may check: valid inputs; invalid inputs; boundary conditions; unusual user actions; error conditions.

How does risk influence Test Technique?

The type of risk may determine the testing method used. Security risk → Security Testing; Heavy user traffic → Performance / Load Testing; Incorrect functions → Functional Testing; Changes affecting existing features → Regression Testing.

How does risk influence Testing Resources?

More time, tools, or experienced testers may be assigned to areas that require greater testing attention.

Define Residual Risk.

refers to the risk that still remains even after testing and other actions have already been performed. It is difficult to completely remove all risks.

What is the goal of managing residual risk?

Testing reduces risk, but it cannot guarantee a completely risk-free system. The goal is to reduce the remaining risk to a level that the organization considers acceptable.

Enumerate the benefits of Risk-Based Testing.

Focus on important areas where failures may cause serious problems; Detecting serious defects earlier by testing important functions first; Select appropriate testing techniques and allocate testing resources efficiently; Improve confidence in important system features before the software is released; Support better decision-making by providing information about the remaining problems and quality of the software.

Define Software Risk Analysis.

A systematic process involving identifying and evaluating any problem that may occur during CREATION, IMPLEMENTATION, and MAINTENANCE of software systems.

What does the presentation say Software Risk Analysis guarantees?

Guarantees that projects are finished on schedule, within budget, and with quality.

Why perform Software Risk Analysis?

Software developers add new features in software development; therefore, software vulnerabilities are introduced.

Enumerate factors that may contribute more to risks.

Time delays; Budget constraints; Lack of Resources; Security.

Define Unknown Unknowns as a scenario of risk occurrence.

Risks that are unknown to the organization and not anticipated. These are unexpected challenges, delays, and problems.

Define Known Unknowns as a scenario of risk occurrence.

Risks that are known but the certainty of occurrence is uncertain.

Define Known Knowns as a scenario of risk occurrence.

Risks that are well understood and documented. These are the risks that are identified early, and teams can plan proper mitigations. These are the manageable risks

Define Technical Risks, including impact.

Description: Risks arising from technical challenges or limitations in the software development process. Impact: Technical risks can lead to delays, cost overruns, and even software failure if not properly managed.

Define Security Risks, including impact.

Description: Risks related to vulnerabilities in the software that could allow unauthorized access or data breaches. Impact: Security risks can lead to financial losses, reputational damage, and legal liabilities.

Define Scalability Risks, including impact.

Description: Risks associated with the software's ability to handle increasing workloads or user demands. Impact: Scalability risks can lead to performance bottlenecks, outages, and lost revenue.

Define Performance Risks, including impact.

Description: Risks related to the software's ability to meet performance expectations in terms of speed, responsiveness, and resource utilization. Impact: Performance risks can lead to user dissatisfaction, lost productivity, and competitive disadvantage.

Define Budgetary Risks, including impact.

Description: Risks associated with exceeding the project's budget or financial constraints. Impact: Budgetary risks can lead to financial strain, project delays, and even cancellation.

Define Contractual & Legal Risks, including impact.

Description: Risks arising from legal or contractual obligations that are not properly understood or managed. Impact: Contractual and legal risks can lead to disputes, delays, and even legal action.

Define Operational Risks, including impact.

Description: Risks associated with the ongoing operation and maintenance of the software system. Impact: Operational risks can lead to downtime, outages, and data loss.

Define Schedule Risks, including impact.

Description: Risks related to delays in the software development process or missed deadlines. Impact: Schedule risks can lead to increased costs, pressure on resources, and missed market opportunities.

(8) Enumerate the Types of Software Risks

Technical risks; Security risks; Scalability risks; Performance risks; Budgetary risks; Contractual & legal risks; Operational risks; Schedule risks.

Define Risk Assessment.

Process of identifying and prioritizing risks at the early stages to avoid loss

What is Risk Identification?

Detecting the type of risk as early as possible and addressing them

What is Risk Analysis in the Risk Assessment section?

Developers analyze the identified risks based on previous software. The Software Development team estimates the probability of risk occurrence

What is Risk Prioritization?

After identifying the risks, the ones with a higher probability of becoming true and higher loss must be prioritized and controlled.

What risk-prioritization formula appears in the presentation?

probability = risk of probability x severity