SQA MIDTERM REVIEWER

The SQA Plan (SQAP)

Deals with the activities and tasks the SQA function is required to carry out over the next year and enables estimating resources required to perform this. The planners of the resource estimates refer to the professional knowledge and experience requirements of the various activities and tasks, and classify the staff requirement accordingly.

The SQAP

is a comprehensive plan that directs the work of the SQA function for a year. It is updated during the year according to the changing and new circumstances.

1. Determining the relevant SQAP outline elements, while considering the needs of each stakeholder in every project performed
2. Preparing the software quality assurance plan (SQAP) according to the standard required elements, considering the stakeholder needs:
3. Finalizing the SQAP according to that agreed with the project managers: It is required to identify activities aimed at reducing or eliminating these risks to determine the expected success of handling these product risks.
4. Identifying and analyzing product risks in the various projects, risks to users of the software products
5. Estimating the SQA function resources required for performing the SQAP: Size of function team, schedule of planned activities, skill and knowledge required, and equipment required.
6. Defining measurements (metrics) to evaluate software quality and for the performance of the SQA function.

The process of preparing an SQAP are:

• Follow up the SQA function's activities and project progress, and perform SQAP revisions required.
• Follow up the changes performed by projects, and perform necessary adaptations to the SQAP.
• Prepare periodic or on demand status reports regarding the progress and findings of the SQAP, and present the information to the organization's quality management and project management.

SQAP Updating Activities

1. SQA Process Implementation Activity Elements
2. Product Assurance Activity Elements
3. Process Assurance Activity Elements

The activity elements are classified into three (3) groups:

• Activities for correcting management deficiencies related to quality issues:

The following are examples of management deficiencies related to quality issues planned to be corrected by the SQA function over the next year:
o Inadequate management quality policy.
o Inadequate activities for establishing a corrective and preventive function in the organization
o Inadequate resources, trained persons, and equipment available to the SQA function
o Inappropriate levels of independence of the SQA function in terms of management and financing.

• Software product risks to users

presents a project list, where for each project, an evaluation of software product risks has to be performed by the SQA function. These include the determination of the risk characteristics and the method in which to handle the product risks.

• Development equipment and tools of software projects

presents a list of projects, where for each project, a list of equipment and tools will be determined based on an analysis of the nature and contract technical requirements of the project.

• Standards, practices, and conventions for software projects

refers to the activity that will evaluate lists of standards, practices, and conventions - applicable to all software projects.

• Resources and schedule estimates for the SQA function

presents resources and schedule estimates for the SQA function activities planned for the next year. The estimates should be based

on the following information derived from the project requirements: staff days, skills and experience, equipment, project activities, type of professional expertise, and schedules.

• Conformance Evaluation of project plans

lists the SQA function's conformance evaluation tasks of the project plan tasks.

• Conformance evaluation of products

lists the evaluations of software development products for their conformance to requirements, standards, and conventions.

• Evaluation of product for acceptability

lists evaluation of the required confidence level of a software project product (including product documentation) to be acceptable by the acquirer that is required to be carried out by the SQA function over the next year.

• Conformance evaluation of product maintenance plan

lists evaluation of conformance of maintenance plans with contract requirements, regulations, standards, and conventions that have to be carried out over the next year.

• Measurement plans for products

lists the measurement plans and required data collection for software products to be prepared by the SQA function over the next year.

• Conformance evaluation of life cycle processes

deals with the evaluation tasks of life cycle processes, models, and procedures to be performed by the SQA function over the next year.

• Conformance evaluation of environment

deals with the evaluation of the adequacy of the environment of development, test, and support services to be performed by the SQA function over the next year.

• Conformance evaluation of subcontractors' participation in project implementation

presents the SQA function evaluation tasks aimed to determine the adequacy of the pre-contract activities.

• Measurement of development, testing, and operation processes

lists the measurement plans and required data collection for software life cycle processes to be prepared by the SQA function over the next year.

• Assessment of staff skills and knowledge requirements and resulting training needs

deals with the evaluation of skills and knowledge required from project staff, identification of deficiencies, and the training program expected to solve these deficiencies.

The Project Plan

Deals with the activities and tasks to be performed by the project team throughout the project's life cycle, namely, during the development and operation stages. Is a comprehensive document that serves the software project throughout the project lifetime the development and operation stages.

The project manager

is usually the person responsible for preparing the project plan - which is expected to be completed and reviewed before the project implementation begins.

• Data collection phase
• Project plan compilation phase
• Project plan reviews
• Project plan updates are usually unavoidable, as change requests originated by the acquirer and methodological and other changes initiated by the project team are very common.

The process of preparing a project plan requires the following phases:

• Data collection phase

includes the study of qualified manpower availability for the project, appropriate development tools, possible development risks, and method or their elimination or at least their mitigation. Consultation with other project managers and experts completes the data collection phase.

development plan and the quality plan.

The elements are classified into two (2) groups:

• Project Products:

A development plan includes the following products:
o Deliverables
o Software products of each activity
o Development process mapping
o Development resources estimation

• Control Methods:

The project manager and the department management control project implementation by defining the monitoring practices to be applied: progress report and coordinating meetings and so on.

• Mapping the Development Process

involves preparing detailed definitions of each of the project's activities. These descriptions include definitions of inputs and outputs and the specific activities planned. Activity descriptions include:
o An estimate of the activity's duration
o The logical sequence in which each activity is to be performed

• Estimating Development Resources:

The type of professional resources required and the estimated quantity are:
o Internal (developer) staff and their professional skills
o External (subcontractor) staff and their professional skills

• Project Staff Organization:

The organization plan includes:
o Organizational structure
o Professional requirements for each team
o Number of team members required for each period of time
o Name of team leaders, and, if possible, the name of team members

• Project Interfaces:

Include:
o Software interface
o Hardware interface
o Teams interface

o Product Risks

a state where the software product may cause damage to the developer and/or to the user of the software.

o Development Risks

a state of a development task or environment, which, if ignored, will increase the likelihood of project failure

• Project Milestones

events of importance in the development process, that is, the completion of the design phase. For each milestone, the completed project products and scheduled times are to be defined.

• Project Cost Estimation

includes human resources costs, subcontractor costs, costs of purchased software, and costs of additional resources, such as travel costs and equipment costs.

• Project Methodology and Development Tools:

The methodology and development tools have to be applied for each phase of the project.

• Software Development Standards and Procedures

are determined by the customer as part of the requirements stated in the project contract.

• Required Development Facilities

include hardware, laboratories, software and hardware development tools, office space, and other items.

• Documentation Control:

The planner is required to define the list of the projects controlled documents and quality records. In addition, a work instruction for the project's documentation control should be prepared.

• Security Including Virus Protection:

The planner is required to define security controls related to the project documents, code in processes, and software products. Special work instructions might be required in certain projects.

• Quality Goals

are the developed software system's substantive quality requirements. These should reflect the major acceptance criteria indicated in the customer's requirement document. These serve as measures of the achievement of the customer's quality requirements.

• Procedures and Work Instructions:

The relevant procedures and work instructions should be defined according to the combined quality assurance and development considerations.

• Criteria for Ending each Project Stage:

A criterion for ending each of the development stages, accepted by the customer and developer, is essential for the regular flow of the development process. It requires:
o The body that applies the criterion
o Defining the criteria: Quantitative criteria are preferred to qualitative ones.

• Project Life Cycle SQA Activities:

The quality plan should provide a complete listing of all planned review activities: design reviews (DRs), design inspections, code inspections, and so on, with the following determined for each activity:
o Planned review activities
o Planned software tests
o Planned acceptance tests for externally developed software

• Configuration Management Tools and Procedures:

The quality plan should specify configuration management tools and procedures, including the timing of baseline version releases.

• Monitoring Measurement Activities:

The planners should define software quality metrics for quality, productivity, schedule keeping, and so forth. The responsibility for performing the measurements and for the monitoring of measurement should be determined.

• Person(s) Responsible for Approving Project Outputs:

The person(s) authorized to approve each of the project products, documents, and code files, especially deliverable items, should be determined.

• Training in Use of New Development Tools:

The need to apply new development tools for given development activities creates a training requirement. The planner should find out which of the scheduled development teams needs training.

• Change Management:

The change management procedures to be applied throughout the project should be defined and agreed upon with the customer.

Project Plans for Small Projects and Internal Projects, hassle, bureaucracy

It is quite natural for project leaders to try to evade the "________" of preparing the development and quality plans. This behavior reflects the tendency to avoid "___________" and the sweeping control that customers may attempt to exercise. This tendency is especially common in two specific situations: small projects and internal projects. The argument for preparing plans for such projects is discussed in the following two sections.

a. Cases/situations where neither development nor quality plans are required, for example, projects requiring 15 man days or less.
b. Cases/situations where the decision to prepare the plans is left to the project leader's discretion, for example, projects that require less than 50 man days, with no significant software risk items identified.
c. A small and complicated project that has to be completed within 30 days, with a heavy penalty for not being completed on time: In this case, partial planning that includes, at least the following, is needed: project mapping of development activities, cost estimates based on resources estimates, and a list of identified project risks, including ways to manage them.

It should be clear that the development and quality plan procedures applicable to large projects cannot be automatically applied to small projects. For these projects, special procedures are needed. These procedures determine how to treat the project in question with respect to the plans:

• Project products
• Project milestones
• Development risks
• Estimates of project costs

Recommended Development Plan for Small Projects:

• Quality goals

Recommended Quality Plan for Small Projects:

• A more comprehensive and thorough understanding of tasks is attained.
• Greater responsibility for meeting obligations may be assigned to project commitments.
• Better understanding with respect to the requirements and schedule may be reached between the developer and the customer.
• It becomes easier for the management and customers to share control of the project and to identify unexpected delays early on.

Advantages to Planned Small Projects

Internal projects

are those projects intended for use by other departments in the organization or by the entire organization, as well as projects dealing with software package development for the software market. The common denominator to all these project types is that no external body participates "as customer" in their development. Internal projects can be on a very large scale.

The Benefits of Full Scale Project Plan for an Internal Project:

The following can enjoy the advantages of plan preparation:

• Software Development Departments
o Avoiding budget overruns: This is important when the profit system center is applied.
o Avoiding damage to other projects caused by delays in releasing professionals occupied in an internal project
o Avoiding loss of market status caused by delayed completion of new software products
o Avoiding loss of market status caused by delayed completion of external projects triggered by late completion of internal projects
• Internal Customers
o Smaller deviations from planned completion dates and smaller budget overruns
o Better control over the development process
o Fewer internal delay damages
• The Organization
o Reduced risk of market loss due to late arrival of software product
o Reduced penalties for noncompliance with contract demands
o Reduced risk of impairing the firm's reputation as a reliable software developer

Cost of Software Quality

is the financial assessment of software quality development and maintenance - is just another class of software quality metrics, where financial values are used as the measuring tool.

• Prevention and appraisal activities budget
• Previous year's failure cost
• Previous project's total quality
• Other department's quality cost

Managerial control over the cost of software quality (CoSQ) is achieved by a comparison of actual performance figures with:

• Control organization initiated costs to prevent and detect software errors.
• Evaluate financial damages of software failures as a basis for revising the SQA budget.
• Evaluate plans to increase/decrease SQA activities, or to invest in new/updated SQA infrastructure.

Cost of Software Quality Measurement Objectives

The classic quality cost model

developed in the early 1950s by Feigenbaum and others, provides a methodology for classifying the costs associated with product quality assurance from a financial point of view. The model was developed to suit quality settings in manufacturing organizations and has since been widely implemented.

• Costs of Control

relate to costs controlled by the software developer and includes the following subclasses:
o Prevention Costs
o Appraisal Costs

• Costs of Failure of Control

relate to the costs of correcting failures that occurred due to unsuccessful prevention activities. The model further subdivides these costs into two subclasses:
o Internal Failure Costs
o External Failure Costs

Prevention Costs

include investments in establishing, updating, and improving a software quality infrastructure, as well as for performing the regular activities required for its operation.

a. Investments in the development of new or improved SQA infrastructure components or regular updating of these components:

o Procedures and work instructions
o Support devices
o Software configuration management system
o Software quality metrics

b. Regular implementation of SQA preventive activities:

o Instruction of new employees on SQA topics and procedures related to their positions
o Instruction of employees in new and updated SQA topics and procedures
o Certification of employees for specific positions
o Analysis of errors and additional data and the subsequent performing of corrective and preventive actions

c. Control of the SQA system through the performance of:

o Internal quality audits
o External quality audits by customers and SQA system certification organizations
o Management quality reviews

Appraisal costs

include the cost of activities performed for a specific project or software system to detect software errors that need to be corrected.

a. Reviews:
o Formal reviews
o Peer reviews
o Expert reviews
b. Cost of software testing
o Unit tests
o Integration tests
o Software system tests
o Acceptance tests
c. External participants:
o Subcontractors
o Software system suppliers
o Project participant

Typical appraisal costs cover:

Internal failure costs

are those incurred through correcting errors that were detected through design reviews, software tests, and acceptance tests performed before the software was installed at customer sites. In other words, internal failure costs represent the cost of error correction subsequent to formal examinations of the software during its development.

a. Cost of rework
b. Cost of design corrections subsequent to design reviews and test findings
c. Cost of correcting programs following test findings
d. Regression tests
e. Domino effect damages

Typical costs of internal failures:

External failure costs

include all costs of correcting failures detected by customers or maintenance teams after the software system has been installed at customer sites. These costs may be further classified into overt external failure costs and hidden external failure costs.

1. Management Control Costs

relate to costs that are controlled by the management and include two subclasses as follows:

o Management Prevention Costs:

Management preventions costs are associated with activities performed to prevent managerial failures or reduce prospects of their occurrence. These activities are required to be performed before work commences on the project and are the responsibility of management. It includes pre-project activities designed to detect erroneous proposal parts and prepare appropriate plans for project performance.

o Management Appraisal Costs:

Management preparations and control costs are associated with activities performed to prevent managerial failures or reduce prospects of their occurrence. Typical management appraisal costs include the costs of activities to control the performance of a specific project.

2. Management Failure of Control Costs

relates to the costs of correcting failures that occurred due to unsuccessful prevention activities. Divides these costs into two subclasses:

o Management Internal Failure Costs:

Management internal failure costs may be incurred throughout the entire course of software development. They are most likely to materialize in connection with failed attempts to identify project risks, failed estimates regarding the appropriate project schedule and budget, as well as to detect in a timely fashion those deviations and problems that necessitate management intervention.

o Management External Failure Costs:

Naturally, most managerial external failure costs incurred after completion of software development and system installation. It includes additional error repairs at customer sites and damages paid to customers due to customer-detected product faults resulting from management commitments and failures.

Application of Cost of Software Quality (CoSQ) System

In the first stage of applying a CoSQ model, the organization should choose its preferred type of cost model-classic or extended. Whichever model is selected, its effectiveness is determined to a great degree by the suitability for the organization of the cost items designed to be measured for the model. In other words, these model items, defined specifically for the organization, are considered relevant to the organization's activities and budget expenditures.

• Definition of a cost of software quality model and an array of cost items specifically for the organization, department, team, or project
• Definition of a method of data collection
• Application of a cost of software quality system, including thorough follow up
• Actions to be taken in response to the findings produced

In order to apply a cost of software quality system in an organization, the following are required:

Application of a CoSQ model

is generally accompanied by problems to be overcome, whatever the industry is.

• Inaccurate and/or incomplete identification and classification of quality costs
• Negligent reporting by team members and others
• Biased reporting of software costs, especially of "censored" internal and external costs
• Biased recording of external failure costs, due to indirect compensation of customers for failures (e.g., discounted future services, delivery of free services), whose implications are not recorded as external failure costs

These impinge upon the accuracy and completeness of quality cost data caused by:

Problems Arising when Collecting Data on Managerial Prevention and Appraisal Costs

• Contract review and progress control activities are performed in many cases in a "part-time mode" and in addition, they are subdivided into several disconnected activities of short duration. The reporting of time invested in these activities is usually inaccurate and often neglected.
• Many of the participants in these activities are senior staff members and are not required to report their time.
• The nature of follow-up activities requiring few hours, and in many cases, even less than an hour, makes them difficult to report accurately.

Problems Encountered in Collection of Data on Managerial Failure Costs

• Determination of responsibility for schedule failures: Schedule failure costs are frequently deliberated for lengthy periods because their direct causes or the specific contributions of each participant to the initial failures are difficult to pinpoint.
• Late payment of customer's overt compensation: At the time of these compensations, it is too late for the effective application of the lessons learned.