Software Quality Assurance Standard Notes

Chapter 12 – Software Quality Assurance Standard

Objectives

  • Introduce Software Quality Assurance (SQA) Standard
  • Discuss the benefits of SQA standard
  • Classifications of SQA standard

Introduction to Software Quality Assurance (SQA) Standard

  • SQA involves monitoring the software engineering processes and methods used to ensure quality.
  • These processes and methods must adhere to one or more standards.
  • Examples of standards include ISO 9000 and CMM/CMMI.

Benefits of SQA Standards

  • Enhances the ability to apply software development and maintenance methodologies and procedures at the highest professional level.
  • Improves mutual understanding and coordination among development teams, especially between development and maintenance teams.
  • Promotes greater cooperation between the software developer and external project participants.
  • Fosters better understanding and cooperation between suppliers and customers, based on adopting known development and maintenance standards as part of the contract.

Organizations Involved in SQA Standards

  • IEEE (Institute of Electrical and Electronics Engineers) Computer Society
  • ISO (International Organization for Standardization)
  • DOD (US Department of Defense)
  • ANSI (American National Standards Institute)
  • IEC (International Electrotechnical Commission)
  • EIA (Electronic Industries Association)

Classification of SQA Standards

1. Quality Management Standards
  • Focus: The organization’s SQA system, infrastructure, and requirements.
  • The choice of methods and tools is left to the organization. (Addresses what process to perform).
  • Compliance ensures software products achieve an acceptable quality level.
  • Includes certification and assessment methodologies.
  • Examples: ISO 9000-3, Capability Maturity Model (CMM).
2. Project Process Standards
  • Focus: Methodologies for carrying out software development and maintenance projects. (Addresses how the process is performed).
  • Defines the steps to be taken, design documentation requirements, contents of design documents, design reviews, review issues, software testing to be performed, testing topics, etc.
  • Many SQA standards in this class can serve as software engineering standards and vice versa.
  • Examples: IEEE Std 1012-1998, ISO/IEC 12207, Capability Maturity Model Integration (CMMI).

Quality Management Standard

  • Certification Standard
  • Assessment Standard
Quality Management Standard: Certification Standard (organization)

*Objectives:
* Enable a software development organization to demonstrate consistent ability to assure that its software products or maintenance services comply with acceptable quality requirements, achieved by certification granted by an external body.
* Serve as an agreed basis for customer and supplier evaluation of the supplier’s quality management system, possibly accomplished by a customer quality audit based on the certification standard’s requirements.

  • Support the software development organization’s efforts to improve quality management system performance and enhance customer satisfaction through compliance with the standard’s requirements.
Quality Management Standard: Assessment Standard (individual in organization)

*Objectives:
* Serve software development and maintenance organizations as a tool for self-assessment of their ability to carry out software development projects.
* Serve as a tool for improvement of development and maintenance processes, indicating directions for process improvements.
* Help purchasing organizations determine the capabilities of potential suppliers.
* Guide training of assessors by delineating qualifications and training program curricula.

Capability Maturity Model (CMM)

  • Developed and promoted by the Software Engineering Institute (SEI), a research and development center sponsored by the U.S. Department of Defense (DOD).
  • CMM is a methodology used to develop and refine an organization's software development process.
  • The model describes a five-level evolutionary path of increasingly organized and systematically more mature processes.
  • CMM establishes a framework for continuous process improvement.
  • It covers practices for planning, engineering, and managing software development and maintenance.
CMM Focus
  • Focus on process improvement.
    • Level 1: Initial - Process is unpredictable, poorly controlled, and reactive.
    • Level 2: Repeatable - Processes are characterized for specific projects and organization is often reactive.
    • Level 3: Defined - Projects tailor their processes from the organization's development methodology.
    • Level 4: Managed - Processes are measured and controlled.
    • Level 5: Optimizing - Focus on continuous process improvement.
CMM's Five Levels of Maturity for Software Processes
  1. Initial: Processes are unpredictable, disorganized, ad hoc, and even confused. Depends on individual efforts and is not considered to be repeatable. This is because processes are not sufficiently defined and documented to enable them to be replicated.
  2. Repeatable: Requisite processes are characterized, established, defined, and documented. As a result, basic project management techniques are established, and successes in key process areas are able to be repeated.
  3. Defined: Organization develops its own standard software development process (methodology). These defined processes enable greater attention to documentation, standardization, and integration.
  4. Managed: Organization monitors and controls its own processes through data collection and analysis.
  5. Optimizing: Processes are constantly improved through monitoring feedback from processes and introducing innovative processes and functionality.
CMM Principle
  • Application of more elaborate management methods based on quantitative approaches increases the organization’s capability to control the quality and improve the productivity of the software development process.
  • The vehicle for enhancement of software development is composed of the five-level capability maturity model.
  • The model enables an organization to evaluate its achievements and determine the efforts needed to reach the next capability level by locating the process areas requiring improvement.
CMM Process Areas
  • Process areas are generic; they define the “what”, not the “how”.
    • Allows use of any life cycle model
    • Allows use of any design methodology, software development tool and programming language
    • Does not specify any particular documentation standard.
CMM Evolution
  1. System Engineering CMM (SE-CMM):
    • Focuses on system engineering practices related to product-oriented customer requirements.
    • Deals with product development: analysis of requirements, design of product systems, management and coordination of the product systems and their integration.
    • Deals with the production of the developed product: planning production lines and their operation.
  2. Trusted CMM (T-CMM):
    • Was developed to serve sensitive and classified software systems that require enhanced software quality assurance.
  3. System Security Engineering CMM (SSE-CMM):
    • Focuses on security aspects of software engineering and deals with secured product development processes, including security of development team members.
  4. People CMM (P-CMM):
    • Deals with human resource development in software organizations: improvement of professional capacities, motivation, organizational structure, etc.
  5. Software Acquisition CMM (SA-CMM):
    • Focuses on special aspects of software acquisition by treating issues – contract tracking, acquisition risk management, quantitative acquisition management, contract performance management, etc. – that touch on software purchased from external organizations.
  6. Integrated Product Development CMM (IPD-CMM):
    • Serves as a framework for integration of development efforts related to every aspect of the product throughout the product life cycle as invested by each department.
CMM Model Levels and Key Process Areas (KPAs)
  • Level 5: Optimizing
    • Process change management
    • Technology change management
    • Defect prevention
  • Level 4: Managed
    • Software quality management
    • Quantitative process management
  • Level 3: Defined
    • Peer reviews
    • Inter-group coordination
    • Software product engineering
    • Integrated software management
    • Training program
    • Organization process definition
    • Organization process focus
  • Level 2: Repeatable
    • Software configuration management
    • Software quality assurance
    • Software subcontract management
    • Software project tracking and oversight
    • Software project planning
  • Level 1: Initial
    • No key process required

Critical Thinking: CMM Disadvantages

  1. Organizations may focus on reaching the next level without truly improving processes.
  2. CMM does not specify how to achieve its goals, requiring flexible thinking based on the organization's processes.
  3. CMM is most effective when implemented early in the software development process, not as an emergency recovery method.
  4. CMM primarily focuses on improving management-related activities, not the software context itself.

Capability Maturity Model Integration (CMMI)

  • Initially tailored to software engineering; later versions became more abstract and generalized for hardware, software, and service development across industries.
  • CMMI capability levels are the same as CMM, with minor changes to levels 2 and 4.
    • Capability maturity level 1: Initial
    • Capability maturity level 2: Managed
    • Capability maturity level 3: Defined
    • Capability maturity level 4: Quantitatively managed
    • Capability maturity level 5: Optimizing
CMMI Focus
  • Focus on CONTINUOUS Process improvement
    • Level 1: Initial - Process unpredictable, poorly Controlled and REACTIVE
    • Level 2: Managed - Process characterized For PROJECTS and is MANAGED
    • Level 3: Defined - Process characterized For the ORGANIZATION And is PROACTIVE
    • Level 4: Quantitatively Managed - Process QUANTITATIVELY Measured and controlled
    • Level 5: Optimizing
CMMI Changes
  • The 18 key process areas (KPAs) of CMM were replaced by 25 process areas (PAs).
  • The PAs are classified by the capability maturity level that the organization is required to successfully perform.
  • For each process area; objectives, specific practices and procedures are defined.
CMMI Process Areas (PA)
  • Level 5 Optimizing
    • Organizational Innovation and Deployment (OID) - Process Mgt
    • Causal Analysis and Resolution (CAR) - Support
  • Level 4 Quantitatively Managed
    • Organizational Process Performance (OPP) - Process Mgt
    • Quantitative Project Management (QPM) - Project Mgt
  • Level 3 Defined
    • Requirements Development (RD) - Engineering
    • Technical Solution (TS) - Engineering
    • Product Integration (PI) - Engineering
    • Verification (VER) - Engineering
    • Validation (VAL) - Engineering
    • Organizational Process Focus (OPF) - Process Mgt
    • Organizational Process Definition (OPD) - Process Mgt
    • Organizational Training (OT) - Process Mgt
    • Integrated Project Management (IPM) - Project Mgt
  • Level 2 Managed
    • Risk Management (RSKM) - Project Mgt
    • Decision Analysis and Resolution (DAR) - Support
    • Integrated Supplier Management (ISM) - Project Mgt
    • Organizational Environment for Integration (OEI) - Support
    • Integrated Teaming (IT) - Project Mgt
    • Requirements Management (REQM) - Engineering
    • Project Planning (PP) - Project Mgt
    • Project Monitoring and Control (PMC) - Project Mgt
    • Supplier Agreement Management (SAM) - Project Mgt
    • Measurement and Analysis (MA) - Support
    • Process and Product Quality Assurance (PPQA) - Support
    • Configuration Management (CM) - Support
      *Level 1 Initial