Software Quality Assurance Comprehensive Study Notes

QUALITY ASSURANCE DEFINITIONS AND SCOPE

• J. M. Juran Definition: J. M. Juran defines Quality Assurance (QA) as the activity of providing to all concerned the evidence needed to establish confidence that the quality function is being performed adequately.
• IEEE Definition: According to the IEEE, quality assurance is defined as "a planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements."
• Product Assurance: This refers to making sure that the final product meets its specifications. This is typically achieved through thorough testing. Ideally, it includes:     * Verifying that requirements are correct.     * Ensuring the design meets those requirements.     * Confirming the implementation reflects the design.
• Process Assurance: This focus is on the process used to create the final product. Key questions include:     * Was the development effort planned?     * Were the plans followed or just ignored?     * Does the development process meet required standards?     * Are best practices utilized?     * QA provides management with objective feedback regarding compliance with approved plans, procedures, standards, and analyses.

SOFTWARE QUALITY ASSURANCE (SQA) AND ERROR TERMINOLOGY

• Donald Reifer's Definition of SQA: The system of methods and procedures used to assure that the software product meets its requirement. This involves planning, measuring, and monitoring developmental activities performed by others.
• Core Focus of SQA: SQA is closely related to errors, but the focus is on preventing and managing them rather than just detecting and fixing them.
• Defect: A situation where the application does not work as per the requirement; the actual and expected results of the software are not in sync. It is an issue in application coding that can affect the whole program.
• Error: A situation occurring when the development team or developer fails to understand a requirement definition, translating that misunderstanding into buggy code. This term is mainly coined by developers.
• Fault: Occurs in software when logic is not incorporated to handle errors in the application. This happens due to factors such as lack of resources or failing to follow proper steps.
• Failure: The accumulation of several defects that lead to software failure, resulting in the loss of information in critical modules and making the system unresponsive.
• Distinction Between Errors and Faults:     * Software Error: A section of code that is incorrect due to grammatical, logical, or other mistakes made by a system analyst, programmer, or developer.     * Software Faults: Software errors that specifically cause the incorrect functioning of the software during a specific application.

CAUSES OF SOFTWARE ERRORS

1. Client-developer communication failures.
2. Deviations from software requirements.
3. Logical design errors.
4. Coding errors.
5. Non-compliance with documentation and coding instructions.
6. Shortcomings of the testing process.
7. Documentation errors.

QUALITY METRICS AND VIEWS

• Development Costs Distribution:     * Req. & Specification: $6\%$     * Design: $5\%$     * Coding: $7\%$     * V&V (Testing): $15\%$     * Maintenance: $67\%$
• Correction Cost Sources:     * Req. & Specification: $56\%$     * Design: $24\%$     * Coding: $10\%$     * Other: $10\%$
• Three Views of Quality:     * Product View: Focuses on functionality, correctness, performance, and usability.     * Process View: Focuses on efficiency, cost and schedule, and failure reduction (avoiding rework).     * Business View: Focuses on timeliness, customer satisfaction, whether it sells, and producing the right product at the right time for the right price.
• Benefits of SQA:     * Produces high-quality software.     * Saves time and costs.     * Improves reliability.     * Beneficial for long-term systems requiring no maintenance.     * Increases company market share.     * Improves the software creation process.     * Cuts maintenance costs.

THE SIX COMPONENTS OF AN SQA SYSTEM

1. Pre-project components: Aimed at improving preparatory steps. Ensures project commitments and goals are defined relative to resources, schedule, and budget.     * Contract Review: Based on functional specifications, budget, and schedule.     * Development and Quality Plans: Prepared once the contract is signed.
2. Software project life cycle components: Divided into two stages:     * Development life cycle stage: Involves Reviews (examining design reports, test documents, installation plans, manuals), Expert Opinion (external professionals support where in-house skill is lacking), and Software Testing (formal SQA aimed at reviewing running software).     * Operation-maintenance stage: Specialized components for functionality and improving maintenance tasks.
3. Infrastructure error prevention and improvement components: Focused on reducing error rates.     * Procedures and work instructions: Generally applicable across the organization.     * Work instructions: Detailed directions for unique instances used by specialized personnel.     * Examples: Templates, checklists, staff training/retraining, certification, preventive/corrective actions, configuration management, and documentation control.
4. Software quality management components: Goals include controlling development and management tasks.     * Project progress control: Detects deviations from plans related to resource usage, schedules, risk management, and budget.
5. Standardization, certification, and SQA assessment components: Responsible for implementing international, professional, and managerial standards.
6. SQA Organizing components: Provides the organizational base (management, testing personnel, SQA units, trustees, committees, and forum members). Objectives include developing SQA components, detecting deviations, and suggesting improvements.

QUALITY ASSURANCE FUNCTION AND OBJECTIVES

• Primary Responsibility: Determining if users' needs have been adequately satisfied.
• Evaluation Areas: Goals, Methods, and Performance.
• QA Objectives- Goals:     * Meet user needs.     * Consistent with needs of other users.     * Consistent with organizational objectives.     * Meet MIS department objectives.     * Consistent with industry and government requirements.     * Controlled and auditable.
• QA Objectives- Methods: Ensures systems use organization and MIS department policies, procedures, standards, and guidelines.     * Policies: Broad course of action selected by the organization.     * Procedures: Particular methods to accomplish tasks.     * Standards: Rules for measuring quantity or quality.     * Guidelines: Recommended methods for performance.
• QA Objectives- Performance:     * Economical: Performing in a way that requires the least cost.     * Effective: Accomplishing results with minimum effort.     * Efficient: Maximizing use of people and machines.

THE QUALITY ASSURANCE PROCESS (STLC)

• Stage 1: Analyze Requirements: QAs analyze functional and non-functional project requirements. Early bug fixing costs less than at production. Duration: $2\text{--}7\,\text{days}$.
• Stage 2: Plan the Tests: Includes testing strategy, scope, budget, deadlines, and tools. Duration: $3\text{--}10\,\text{days}$.
• Stage 3: Design the Tests: Creation of test cases and checklists. Testers compare actual results with expected results. Duration: $2\text{--}5\,\text{days}$.
• Stage 4: Execute Tests and Report Defects: Starts with unit tests by developers, followed by QA team testing at API and UI levels. Bugs recorded in defect-tracking systems. Duration: $5\text{--}2\,\text{days}$.
• Stage 5: Run Re-Tests and Regression Tests: Re-testing resolved bugs and performing regression tests to ensure fixes didn't impact existing functions. Duration: $3\text{--}15\,\text{days}$.
• Stage 6: Run Release Tests: Identification of affected functions after release notification. Includes smoke tests for stability and modified test suites. Duration: $2\text{--}7\,\text{days}$.

PRINCIPLES, ROLES, AND QUALITIES OF THE QA GROUP

• Three General Principles:     1. Know what you are doing: Understand what is being built and how, including milestones and tracking.     2. Know what you should be doing: Having explicit requirements, specifications, and acceptance tests.     3. Know how to measure the difference: Using formal methods (math properties), testing (explicit inputs), inspections (checklists for design/code), and metrics.
• QA Team Roles:     * QA Engineer: Tests software and identifies bugs.     * QA Test Analyst: Plans and documents processes.     * QA Team Lead: Supervises the team.     * QA Manager: Communicates with the client.     * Test Architect: Develops the tests.
• QA Group Activities (Pyramid Model Structure):     * Analyze Errors (Foundation).     * Develop Standards and Guidelines.     * Review and Build a Quality Environment.     * Provide Technical Advice.     * Review Applications (Top).

DETAILED QA ACTIVITIES AND CHALLENGES

• Analyze Errors: Providing a basis for improvement. QA issues "error alerts" to help programmers avoid repeating mistakes and publicize QA services.
• Developing Standards & Guidelines: Includes System development process, Documentation, Programming, Control (program monitoring), Operating (production mode), Hardware and Software, and Performance standards.
• Review & Build a Quality Environment: Reviewing general controls (like differentiation of duties/job descriptions) for adequacy.
• Provide Technical Advice: Assistance in understanding policies, utilizing hardware/software, and assessing impacts on other systems. Forms include answering specific requests or volunteer advice.
• Review Application: Six-part review including system cost, user needs, consistency with MIS goals, compliance with organization policies, economic use of resources, adequacy of the testing plan, and conversion plans.
• ERP Department Challenges & QA Responses:     * Challenge: Executive/User ignorance; QA Task: Evaluate systems in all phases and supply technical assessment.     * Challenge: Changing technology; QA Task: Acts as internal/technical consultant; knows current technology.     * Challenge: Evaluating products; QA Task: Build quality environment and counsel management.     * Challenge: Few standards; QA Task: Help set and evaluate standards.     * Challenge: Knowing magnitude of problems; QA Task: Quantify and identify problems; determine error costs.

TECHNIQUES FOR PROCESS ASSURANCE

• Affinity Diagrams: Sorting ideas or issues into related groups under headings.     * Process: Team identifies issues (brainstorming), records on cards, lays them out randomly, groups them by consensus, describes the grouping, and identifies relationships.     * Example Groups: Conditions of Service (pay, benefits), Environment, Location, Decoration, Workload (shortages, distribution), and Staff (gossip, unhelpful colleagues).
• Brainstorming: Generating many ideas from a group in a short period.     * Steps: Write problem on board, choose leader, assemble group, members give ideas (rotation or random), write all ideas, then eliminate and prioritize.     * Rules:         1. All comments/ideas are equally valid.         2. All ideas must be recorded regardless of absurdity.         3. No criticism during the generation phase.         4. Think the unthinkable.