Chapter 11 – Reliability Engineering (Ian Sommerville, 10th Ed.)

A set of question-and-answer flashcards covering definitions, metrics, requirements, architectures, dependable programming practices, and measurement techniques from Chapter 11 (Reliability Engineering).

What is the definition of software reliability?

The probability of failure-free system operation for a specified time, in a given environment, for a given purpose.

How is availability defined?

The probability that a system, at a point in time, is operational and able to deliver requested services.

What does an availability of 0.999 mean?

The system is up and running 99.9 % of the time (about 84 s downtime per 24 h).

In reliability engineering, what is a human error or mistake?

Human behaviour that introduces faults into a system during development or operation.

Define a system fault.

A characteristic of a software system that can lead to a system error (e.g., defective code).

What is a system error?

An erroneous system state that can lead to behaviour unexpected by users.

Define system failure.

An event where the system does not deliver the service expected by its users at some point in time.

Do all faults cause failures? Why or why not?

No. Faulty code may never be executed or an erroneous state may be corrected before failure occurs.

Name the three broad strategies for reliability achievement.

Fault avoidance, fault detection & removal, and fault tolerance.

What is fault avoidance?

Developing a system so that human error is avoided and system faults are minimised before delivery.

Explain fault tolerance.

Designing the system so that faults do not lead to system errors or errors do not cause failures during operation.

Why are the costs of residual fault removal high late in development?

Because fewer faults remain and each is harder to find and fix, increasing cost per detected error.

Why can reliability be formally defined only with respect to a specification?

Because a failure is a deviation from the specified behaviour; without a spec, failure cannot be objectively stated.

Why is perceived reliability often more important than formal reliability?

Users rarely read specifications and judge reliability based on their experience and the consequences of failures.

Give two factors that simple availability percentages ignore.

(1) Number of users affected by an outage; (2) Length of the outage.

Why does removing X % of faults not guarantee X % reliability improvement?

Remaining faults may reside in frequently executed code, while removed faults may be in rarely used paths.

List the three main reliability metrics.

Probability of Failure on Demand (POFOD), Rate Of Occurrence Of Failures (ROCOF)/Mean Time To Failure (MTTF), and Availability (AVAIL).

When is POFOD the most appropriate metric?

For systems with intermittent, infrequent demands where failures have serious consequences (e.g., emergency shutdown).

Interpret a ROCOF value of 0.002.

About 2 failures are expected in 1000 operational time units (e.g., hours).

What is the relationship between ROCOF and MTTF?

MTTF is the reciprocal of ROCOF (MTTF = 1 ⁄ ROCOF).

For which kind of systems is high availability (e.g., 0.998+) most relevant?

Continuously running, non-stop systems like telephone switching or railway signalling.

State one benefit of specifying quantitative reliability requirements.

They provide an objective criterion for deciding when to stop testing.

Why might ATM designers prioritise availability over reliability?

Database mechanisms can correct transaction problems; keeping machines operational for customers is more critical.

What availability was specified for an ATM network’s database service?

About 0.9999 between 7 am and 11 pm (≈ <1 min downtime per week).

What is an example POFOD requirement for an insulin pump transient failure?

POFOD ≈ 0.002 – no more than 1 failure in 500 demands that users can correct by recalibration.

Name the four categories of functional reliability requirements.

Checking, Recovery, Redundancy, and Process requirements.

Give an example of a redundancy requirement from the notes.

"Copies of the patient database shall be maintained on two separate servers not housed in the same building."

What is the purpose of a protection system?

To independently monitor a controlled system and take emergency action (e.g., shut-down) if a problem is detected.

Why should protection systems be diverse?

To avoid common-mode failures by using different technologies from the control system.

Describe a self-monitoring (multi-channel) architecture.

Multiple diverse channels perform the same computation; outputs are compared, and discrepancies raise a failure exception.

How many computers are in the Airbus flight-control system, and why use diversity?

Five; diversity of processors, chipsets, languages, and teams reduces common-mode failures.

What is N-version programming?

Running an odd number (usually three) of independently developed software versions in parallel and voting on results.

Why can design diversity fail to deliver full independence?

Teams share similar cultures and may misinterpret specifications in the same way, leading to common errors.

State two strategies to promote software diversity.

Use different programming languages and different algorithms/design methods.

List any four dependable programming guidelines.

(1) Limit visibility of information; (2) Check all inputs; (3) Provide handlers for all exceptions; (4) Minimise error-prone constructs; (5) Provide restart capabilities; (6) Check array bounds; (7) Include timeouts when calling external components; (8) Name all real-world constants.

Why should visibility of data be limited in a program?

To prevent accidental corruption of program state by components that don't need access.

Give two examples of validity checks on inputs.

Range checks and representation checks (e.g., no numerals in a name field).

Name three error-prone language constructs.

Unconditional branches (goto), unchecked pointers, and dynamic memory allocation (others include recursion, parallelism, interrupts, etc.).

Why include timeouts when calling external components?

To detect ‘silent’ failures of remote services and allow the system to recover after a defined period.

What operational data is needed to measure POFOD?

Number of service requests and number of failures on those requests.

Why is statistical (reliability) testing separate from defect testing?

It requires input data reflecting real operational profiles, whereas defect testing often uses atypical or extreme inputs.

Define an operational profile.

A test data set whose input frequencies match the expected normal usage distribution of the system.

List two problems in reliability measurement.

(1) Operational profile may not match real use; (2) Highly reliable systems rarely fail, so many tests are needed for statistical significance.

Summarise the three pillars of software reliability from the key points.

Fault avoidance, fault detection/removal, and fault tolerance.