Quality Assurance and Management Study Notes

Unit 7: Quality Assurance and Management

Introduction

• Prepared By: Prof. Neha Chauhan

• Subject: Software Engineering

• Subject code: 102045602

• Institute: A.D. PATEL INSTITUTE OF TECHNOLOGY (A Constituent College of CVM University)

• Department: Information Technology

Contents

• Quality Concepts

• Software Quality Assurance

• Software Reviews (Formal Technical Reviews)

• Software Reliability

• The Quality Standards: ISO 9000, CMM

• Six Sigma for Software Engineering

• SQA Plan

Quality Concepts

• Software Quality Issue:
    - Blame Dynamics:
      - Customers blame developers for low-quality software citing sloppy practices.
      - Developers blame customers for unrealistic delivery timelines and constant changes, impacting validation time.

Software Quality

• Definition:
    - Software quality is defined as:
      - An effective software process that creates a useful product providing measurable value for producers and users.

Stakeholder Responsibility in Software Quality

• Involvement:
    - Everyone (Software Engineers, Managers, Stakeholders) holds accountability for product quality.

• Importance of Quality Focus:
    - A strong quality focus reduces rework, resulting in lower costs and improved time-to-market.

Steps to Achieve High-Quality Software

1. Proven Software Engineering Process & Practice:
      - Establishing reliable methodologies to apply throughout the software lifecycle.

2. Solid Project Management:
      - Effective management of project scope, timelines, and resources.

3. Comprehensive Quality Control:
      - Implementing systems for ongoing quality checks throughout development.

4. Quality Assurance Infrastructure:
      - Establishing support systems for maintaining quality standards.

Work Product Quality

• Definition:
    - A software product that meets customer needs, performs accurately, and reliably provides user value.

• Quality Monitoring:
    - Quality is tracked via quality control activities and measured by analyzing pre-delivery errors and post-release defects.

Quality Perspectives

1. Transcendental View:
      - Quality is recognized intuitively but not explicitly definable.

2. User View:
      - Quality is defined through specific user goals; a product exhibits quality if it meets these goals.

3. Manufacturer’s View:
      - Defined by conformity to original specifications; alignment with specifications indicates quality.

4. Product View:
      - Quality relates to inherent characteristics of the product (functions/features).

5. Value-Based View:
      - Measured by what customers are willing to pay; effective quality combines all the views above.

Types of Software Quality

• Quality of Design:
    - Encompasses requirements, specifications, and system design.

• Quality of Conformance:
    - Focuses on adherence to design during implementation.

• User Satisfaction Factors:
    -
      - Compliance to product requirements.
      - Quality attributes.
      - Timely delivery within budget.

Effective Software Process

• Infrastructure Support:
    - An effective process provides structure for creating high-quality software.

• Management Aspects:
    - Create checks and balances to avoid project chaos.

• SW Engineering Practices:
    - Enable problem analysis and sound design solutions.

• Umbrella Activities:
    - Include change management and technical reviews, critical for quality assurance.

Useful Product Characteristics

• Delivery of Core Features:
    - A useful software product must meet specified content, functions, and features while ensuring reliability and error-free operation.

• Requirement Satisfaction:
    - Both explicit and implicit requirements (such as ease of use) must be satisfied.

Value Addition in Software Quality

• Benefits for Producers:
    - High-quality software incurs less maintenance, fewer bug fixes, and reduced support needs.

• Benefits for End-Users:
    - Delivers capabilities that enhance business processes efficiently.

• Outcome:
    - Results in:
      1. Increased software product revenue.
      2. Improved profitability supporting business processes.
      3. Enhanced availability of crucial information.

Quality Dimensions (David Garvin)

1. Performance Quality:
      - Measure of whether software delivers specified content, functions, and features effectively.

2. Feature Quality:
      - Evaluation of unexpected and delighted features for users.

3. Reliability:
      - Assurance of feature delivery without failures; consistently available and error-free functionality.

4. Conformance:
      - Adherence to local and external software standards; evaluation of user interface against design rules.

5. Durability:
      - Capability of maintenance and debugging without introducing new issues.

6. Serviceability:
      - Maintenance time sufficiency and availability of information for support staff.

7. Aesthetics:
      - Qualitative aspect of software presence, elegance.

8. Perception:
      - Personal biases may influence perceived quality levels.

Software Quality Dilemma

• Challenge of Quality Balance:
    - Poor Quality Consequences:
      - A poorly developed product fails to attract buyers.
    - Perfectionism Costs:
      - Excessive effort in perfecting software leads to resource exhaustion, potential market loss, and bankruptcy.

• Aim for Balance:
    - Achieve an acceptable level of quality without overextending resources or missing market opportunities.

Cost of Quality

• Prevention Costs Include:
    - Quality planning.
    - Formal technical reviews.
    - Test equipment and training.

• Internal Failure Costs Include:
    - Rework, repairs, and failure mode analysis.

• External Failure Costs Include:
    - Complaint resolutions, returns, help line support, warranty work.

Achieving Software Quality

• Critical Success Factors:
    - Software Engineering Methods.
    - Project Management Techniques.
    - Quality Control.
    - Quality Assurance.

Software Quality Assurance (SQA)

• Definition:
    - Software quality assurance (SQA) is an umbrella activity applied throughout the software process to provide high confidence in the product's quality.

• Components of SQA:
    - SQA process.
    - Specific quality assurance and quality control tasks.
    - Effective engineering practices.
    - Control of software work products and changes.
    - Compliance procedures with development standards.
    - Measurement and reporting mechanisms.

Importance of SQA

• Quality control and assurance are crucial in producing reliable software for external use.

• Historically, prior to the 20th century, quality control was the job of individual craftspeople creating products.

• With the advent of mass production, quality control transitioned to separate roles.

• SQA is critical from a project’s inception, starting in the analysis phase.

SQA Practices

• Checks and Compliance:
    - SQA ensures adherence to software standards and processes.
    - Evaluates compliance through process monitoring, product assessment, and project management.
    - Aims to deliver a final product conforming to requirements and standards.

Elements of SQA

• Key Elements Include:
    - Standards.
    - Reviews and audits.
    - Testing.
    - Error/defect collection and analysis.
    - Change management.
    - Education.
    - Vendor management.
    - Security and safety management.
    - Risk management.

Role of SQA

• SQA Plan Preparation:
    - Develops SQA plans identifying evaluations, audits, applicable standards, and procedures for errors.

• Participation in Software Process Development:
    - Reviews process descriptions for compliance with organizational and external standards.

• Review Activities:
    - Monitors software engineering activities and ensures process compliance, tracking deviations.
    - Conducts audits of software work products and reports results to project management.
    - Records noncompliance and monitors until resolution.

SQA Goals

1. Requirements Quality:
      - Influences quality of subsequent work products.

2. Design Quality:
      - Assesses design model elements for quality and conformity.

3. Code Quality:
      - Ensures source and related work products comply with coding standards for maintainability.

4. Quality Control Effectiveness:
      - Deploys resources appropriately to maximize quality outcomes.

Statistical QA

• Steps for Statistical Quality Assurance:
    1. Collect and categorize defect information.
    2. Trace defects to root causes (e.g., specification non-conformance, design errors).
    3. Apply the Pareto principle to isolate the "vital few" causes (80/20 rule).
    4. Implement corrective actions for identified root causes.

Common Causes of Defects

• Defect Categories Include:
    - Incomplete or erroneous specifications (IES).
    - Miscommunication with customers (MCC).
    - Intentional specification deviations (IDS).
    - Programming standard violations (VPS).
    - Errors in data representation (EDR).
    - Inconsistent component interfaces (ICI).
    - Design logic errors (EDL).
    - Incomplete or erroneous testing (IET).
    - Incomplete or incorrect documentation (IID).
    - Errors in programming language translation of design (PLT).
    - Ambiguities in human/computer interfaces (HCI).
    - Miscellaneous defects (MIS).

Software Reviews (Formal Technical Reviews)

• Definition:
    - A formal technical review (FTR) is a quality control activity performed by engineers to evaluate software work products.

• Objectives of an FTR:
    1. Uncover errors in functionality, logic, or implementation of software.
    2. Validate requirements compliance of the software under review.
    3. Ensure conformity to predefined standards.
    4. Foster uniform development practices.
    5. Enhance project manageability.

Review Reporting and Record Keeping

• During FTRs:
    - A designated reviewer records all raised issues.
    - Post-review, issues are summarized into a review issues list, concluded with a formal summary report.

Review Guidelines

• Establishing Guidelines for FTRs:
    - Must be defined, distributed, and adhered to by all reviewers.
    - Prioritize product assessment over personal roles.
    - Maintain an agenda, limit debates, and focus on problem identification without immediate solutions.
    - Keep participant numbers limited and prepare them in advance.
    - Utilize checklists and ensure adequate resources are allocated for reviews.
    - Provide effective reviewer training and reflect on previous reviews to improve processes.

Sample-Driven Reviews

• Considerations:
    - Ideal practice involves every product undergoing FTRs, but limited resources may lead to review omissions despite recognized value.

Software Reliability

• Definition:
    - Software reliability is statistically defined as "the probability of failure-free operation of a computer program in a specified environment for a specified time."

Measures of Reliability

• Simple Reliability Measure:
    - Mean-Time-Between-Failure (MTBF):
      - $MTBF = MTTF + MTTR$ where:
        - MTTF: Mean-Time-To-Failure
        - MTTR: Mean-Time-To-Repair
    - Emphasis on MTBF over total defect count because users care about failures rather than defect numbers.
    - Failures-In-Time (FIT):
      - Statistical measure of failures per billion hours of operation.

Software Safety

• Definition:
    - Software safety focuses on identifying and assessing potential hazards affecting software. Early recognition allows for design adjustments to eliminate risks.
    - Distinction from reliability: Reliability assesses likelihood of failure, while safety analyzes failures that might lead to hazards.

Quality Standards

• ISO 9001 Overview:
    - Organizations adopt quality assurance systems to enhance product/service quality.
    - ISO Standards Origin:
      - Established by the International Organization for Standardization (ISO) in Switzerland.
    - Documentation Necessity:
      - Critical aspect of an ISO 9001 Quality Management System.
    - Applies to all engineering disciplines including software engineering.

ISO 9001 Requirements

• Topics covered by ISO 9001:2000 include:
    - Management responsibility.
    - Quality system establishment.
    - Contract reviews.
    - Design control.
    - Document and data control.
    - Product identification.
    - Traceability and process control.
    - Inspection and testing protocols.
    - Preventive actions and quality records controls.
    - Internal quality audits, training, servicing, statistical techniques.

ISO Certification Process

• To achieve ISO 9001:2000 registration, organizations must:
    - Define policies and procedures per the requirements.
    - Demonstrate adherence to these policies and procedures.

Six Sigma Overview

• Definition:
    - Six Sigma is a quantitative improvement approach applicable to any process.
    - It advocates eliminating errors across various processes from manufacturing to services.
    - Defect Benchmarking:
      - Achieving Six Sigma means not more than 3.4 defects per million opportunities.
      - Comparative defect levels for other Sigma levels:
        - 5 Sigma: 230 defects per million
        - 4 Sigma: 6210 defects per million

Six Sigma Methodologies

1. DMAIC (Define, Measure, Analyze, Improve, Control):
      - Define: Identify the problem or process for improvement related to customer goals.
      - Measure: Decide on a systematic measurement of the process.
      - Analyze: Explore the process to identify root causes of the problems.
      - Improve: Present and implement solutions for identified issues.
      - Control: Use Statistical Process Control to ensure continuous improvement.
      
      Tools are available for assessing yield and defects per million opportunities.

2. DMADV (Define, Measure, Analyze, Design, Verify):
      - Similar initial stages as DMAIC.
      - Design: Innovate to avoid defect root causes and meet requirements.
      - Verify: Compare the new design with standards to identify gaps.

Capability Maturity Model (CMM)

• Purpose:
    - Evaluate organizational process maturity for software engineering practices.

• Process Maturity Levels Defined by SEI:
    - Level 1: Initial:
      - Ad-hoc, chaotic processes with success reliant on individual efforts.
    - Level 2: Repeatable:
      - Basic project management established to manage cost, schedule, and performance for similar applications.
    - Level 3: Defined:
      - Documented, standard processes integrated organization-wide, all projects follow approved processes.

• Level 4: Managed:
    - Detailed quality measures collected, processes and products quantitatively understood and controlled.

• Level 5: Optimizing:
    - Continuous improvement fostered using quantitative feedback and innovative testing.

Software Quality Assurance (SQA) Plan

• Purpose:
    - The SQA Plan serves as a roadmap for instituting software quality assurance practices.

• Development:
    - Produced by either the SQA group or the software team if no dedicated group exists.

SQA Plan Structure Recommendations

1. Purpose and scope of the plan.

2. Descriptions of all software engineering work products involved.

3. Applicable standards/practices during the software process.

4. SQA actions/tasks placement throughout the process.

5. Tools and methods aiding SQA actions and tasks.

6. Software configuration management procedures.

7. Methods for maintaining SQA records.

8. Defined organizational roles and responsibilities concerning quality assurance.