SENG 438 Software Testing, Reliability and Quality - In Depth Notes

Quality in Software Development Process

  • Incorporating quality concerns:
    • Requirement, architecture, design, implementation, testing, release, and maintenance are essential phases.
    • Architectural analysis and quality attributes can be assessed using methods like ATAM, CBAM.
    • Software Quality Assessment Method: RAM.

Software Quality System (SQS) and Software Quality Assurance (SQA)

  • Reliable Software:
    • Runs correctly and consistently, fewer defects, manages exceptions effectively, easier installation.
    • Remaining defects should not disrupt normal operation or cause destructive damage.
  • Developing Reliable Software Requires:
    • Establishment of SQS & SQA programs.
    • Implementation of Software Reliability Engineering (SRE) processes.
Goals of SQS:
  • Incorporating quality from the outset of development.
  • Continuous tracking of quality throughout the software lifecycle.

Software Quality Assurance (SQA)

  • A structured approach ensuring that software processes and products conform to standards.
  • Goals: Improve quality by monitoring adherence to standards and procedures.
  • Steps to Establish SQA Program:
    • Management support and alignment on goals.
    • Identify issues, develop a plan, set standards, establish functions, and evaluate outcomes.

Quality Plans

  • Quality plans help improve predictability and control of software qualities.
  • They prevent misunderstandings and enhance project effectiveness.
    • Includes Test Plans and Data Plans.
  • Quality Plan Guidelines:
    • Requirement and specification management, development and documentation management, design evaluations, and acceptance processes.

Data (Measurement) Plan

  • Prescribes measurements during a project:
    • What to measure, methods of collation, and interpretation of data.
  • Data should feed into a higher-level collection and application system.

Test Plans

  • A comprehensive plan to manage and ensure effectiveness of testing activities.
  • Components of a Test Plan:
    • Objectives, background, scope, test items, and features tested.
    • Testing approaches covering various types (security, performance, regression).
    • Pass/fail criteria, suspension/resumption criteria, deliverables, and resources.

Sample SQS Plan Elements

  1. Purpose: Clarifying the goals of the plan.
  2. Management: Organization, tasks, and responsibilities.
  3. Documentation: Outlining necessary documentation.
  4. Standards: Establishing coding, testing standards, and performance metrics.
  5. Reviews: Audit processes to ensure compliance with set standards.
  6. Testing: Structured approaches for testing phases and problem reporting.

Post Release Activities

  • Focuses on activities following software delivery:
    1. Staffing Needs: Aligning customer and supplier staff for issue resolution.
    2. Monitoring Reliability: Systematic collection of failure data post-release.
    3. Customer Satisfaction Tracking: Surveying customers to assess reliability perceptions.
    4. Feature Management: Introducing new features judiciously to maintain reliability.
    5. Process Improvement: Root-cause analysis on faults to enhance future practices.
Future Directions
  • Evolving testing practices to address "Quality at Speed" amid growing system complexity.
  • Embrace DevOps and AI both for testing and being tested.

Conclusions

  • Implementing effective software quality programming is complex but essential.
  • Mechanisms for data collection and analysis are crucial along with fault identification and remedial strategies.
  • Organizational readiness, quality-oriented mindset, and thorough training are vital for maintaining high quality standards.