Quality control & Assessment

What is quality assurance (QA)? What 3 phases does it include in a lab?

QA: the broad system used to support quality throughout the testing process

• Pre-analytical phase

• Analytical phase

• Post-analytical phase

What does CLIA ‘88 establish for QA? What does 3 things does this ensure about lab results?

CLIA ‘88 establishes: minimum federal requirements for QA programs

• Ensures lab results are: accurate, timely, and reliable

  • Part of Qa (quality assessment) & QC (quality control)

What is quality assessment and what is included in it? What is quality control and what does it help determine?

Quality assessment: Evaluates effectiveness of lab policies, processes & services across the total testing process (pre, analytical, & post)

• Includes: personnel, test performance, equipment, testing & reporting, complaints & corrective actions

Quality control: Lab procedures that monitor the performance of a particular procedure or test to assure the accuracy of measurements

• Helps determine if a test system is operating within acceptable limits

What is used (quality assessment or quality control) to ensure the reliability of results? What does the reliability of procedures and tests depend on?

Quality assessment ensures reliability of results.

Combo of:

• Accuracy

• Precision

• Sensitivity

• Specificity

• Predictive values (NPV and PPV)

• Efficiency

What is accuracy? What does it imply?

How close a test result is to the actual value

• Use reference samples/standards that have known values to check accuracy of samples

  • Implies freedom from error

What is precision? What does it imply?

How close the test results are to one another

• The reproducibility of test results

• Implies freedom from variation

What is a confusion matrix?

A structured performance table used to evaluate the diagnostic accuracy of a lab test by comparing the test’s results against a known clinical gold standard (true disease status of a patient)

The confusion matrix consists of what 4 things?

1. True-positive (TP): test result correctly indicates presence of disease

2. True-negative (TN): test result correctly indicates absence of disease

3. False-positive (FP): test result that wrongly indicates the presence of a disease

4. False-negative (FN): test result that wrongly indicates absences of a disease

What is diagnostic sensitivity? Analytic sensitivity?

Diagnostic sensitivity: Ability of a test to correctly identify if someone has a disease

• ability of an assay to correctly predict with a positive result

Analytic sensitivity: Lowest amount of an analyte that a test can detect

What is diagnostic specificity? Analytic specificity?

Diagnostic specificity: Ability of a test to correctly identify if someone does not have the disease

• ability of an assay to correctly predict with a negative result

Analytic specificity: Ability of a test to only detect the analyte of interest

What is positive-predictive value (PPV)?

Indicates the probability that a positive test result indicates the presence of the disease

What is negative-predicative value (NPV)?

Indicates the probability that a negative test result indicates the absence of a disease

What is efficiency?

The fraction of all the tested individuals who were correctly classified as either having or not having the disease

What is the formula for Sensitivity, specificity, PPV, NPV, and Efficiency?

Multiply all by 100 to get them in %

Sensitivity: T.P / T.P + F.N

Specificity: T.N / T. P + T.N

PPV: T.P / T.P + F.P

NPV: T.N / T.N + F.N

Efficiency: T.P + T.N / total

Quality assessment also includes guidelines for clinical tests. What 3 guidelines are there? Briefly describe them. What will be the sensitivity and specificity for each guideline?

1. Screening test: identifies asymptomatic/symptomatic individuals who might have the disease

• High sensitivity (90%+)

• Moderate specificity (80-89%)

2. Diagnostic test: confirmatory test that determines the presence/absence of a disease to establish diagnosis

• Moderate sensitivity (80-89%)

• High specificity (90%+)

3. Gold standard: best test available (may be invasive or expensive)

• High sensitivity & specificity

How do you determine if a test is better as a screening test or a diagnostic test?

Screening: needs high sensitivity → SnNout (sensitivity negative = rule out)

Diagnostic (confirmatory): needs high specificity → SpPin (specificity Positive = ruling in)

What is quality control (QC) used to do? What does it measure over time? During what lab phase does QC span over? What two parts of QC are there?

Used to evaluate the analytical testing process to detect examination errors before releasing patient results → reduces & corrects errors

• Measures: accuracy, precision & reproducibility

• Analytical phase

2 parts: Internal & external

What is internal quality control? How many tests need to be run at a minimum according to CLIA?

• Evaluates daily measurements → provided by outside vendor

  • Need to run a positive & negative control

Run tests at a minimum of once per day

What is quality control statistics? What is part of the statistics?

QC stats are used to evaluate quality control.

Stats includes:

• Mean, Median (middle ranked value), Mode (most recurring # in data)

• Standard deviation → describes distribution of all values around the average

• Coefficient of variation (CV) → relationship of the SD to the mean

  • Measures precision → lower CV% the more precise

• Control range → testing healthy people in a population (for reference)

What is the formula for coefficient of variation (CV)?

%CV = (SD * 100) / mean

remember: lower CV = higher precision

How many individuals does CLSI recommend you test in a population to determine the control range?

What is the acceptable control range (confidence interval - CI)?

CLSI recommends: 120 individuals

Acceptable range: 95% confidence interval (2SD)

• 95% CI = mean +- 2SD

What is a quality control chart? What is the most commonly used chart?

A plot of the daily control results to detect QC errors & a visual documentation of control values

Common chart: Levey-Jennings Chart

What specifically does a Levey-Jennings Chart monitor? What should be noted on the chart and how many measurement are needed for calculation?

Monitors: precision & stability of lab analytical processes over time

Noted: Mean & SD w/ at least 20 measurements

Levey-Jennings charts can have random or systematic errors. What are random errors? What does it directly measure for a lab and can it ever be 0% error?

What is a specific type of random error that can also occur?

Random errors: unpredictable variations that cause data points to scatter on both sides of a target value

• measures lab assay’s precision (reproducibility)

• Will never be 0%, but can never be far from 0 (~5%)

Dispersion error: wide scattering of control values

• Caused by: bubbles in reagent line, inconsistent sample mixing, system failure, variability between operators

Levey-Jennings charts can have random or systematic errors. What are systematic errors? What do these errors impact in a lab and what causes these errors?

Some error types within this are shifts and trends/drifts. Describe them.

Can errors be fixed?

Systematic errors: predictable & continuous variations that cause all measurements to skew uniformly in one direction (consistently too high or too low)

• Impacts lab assay’s accuracy

• Causes: miscalibration of testing instrument, reagent & material degradation, mechanical & instrumental wear

Shifts: rapid & sustained change away from the mean in one direction for 5-7 days

• Causes: malfunction, changing reagent lot w/out recalibration, wrong controls

Trends/drifts: gradual change away from the mean in control sample result for 3 days

• Causes: deteriorating reagents, diminishing light source, corrosion of electrodes, accumulation of debris

What is total analytic error (TAE)?

• Estimates the combined effect of all error sources that’s affecting a single lab measurement

  • Analyzers do not suffer from one error type at a time

What is allowable total error (ATE)?

• Amount of error allowed w/out invalidating the interpretation of a test result

  • Normally up to 2 SD

TAE < ATE

Internal QC also relies on Westgard Rules for Levey-Jennings chart. How many rules need to be applied to detect random and systematic errors?

Need to apply 2 rules

Westgard rules are used on Levey-Jennings chart. What does a 1-2s, 1-3s, 2-2s, R-4s, 4-1s, and 10x mean?

1-2s violation is a warning rule: 1 control (point) exceeds 2 SD

1-3s violation is a rejection: 1 control (point) exceeds the mean at 3 SD

2-2s violation is a rejection: 2 controls (points) are at 2 SD twice in a row

R-4s violation is a rejection: 1 control in the same run exceeds a 2 SD and a -2 SD

4-1s violation is a rejection: 4 controls fall either all above the +1 SD line or all below the -1 SD line

10x violation is a rejection: 10 controls all fall on one side of the mean

How do you correct a Levey-Jennings chart when it has violations?

Accept results if it’s a warning

Reject if it’s a violated mandatory rule

• Repeat QC test with new reagent

• Review procedures

• Search for recent events (for potential change)

• Prepare new controls

• Follow troubleshooting guide

• Contact manufacturer

Quality control also has an external QC. What is this? What does CLIA ‘88 require?

Means by which QC between labs is maintained.

CLIA requires proficiency testing (PT) programs.

• Administered by CMS approved agency (ex. CAP)

What are the 2 types of proficiency testing that falls under external QC?

1. Commutable PT samples (preferred): “true” value assigned to sample, mean & SD used to grade results

2. Noncommutable PT samples: “peer group” is all labs using same method, mean & SD of labs are used to grade results

What is the proficiency testing process and grading?

Process:

1. Labs receive 3 surveys a year → each survey has 5 samples

2. Lab performs indicated testing → treat as if normal patient sample

3. Results reported to surveying agency

4. Samples are graded based on sample administered

Grading:

• Acceptable: 2 SD of mean

  • Improvement needed: 2-3 SD of mean

  • Unacceptable: >3 SD of mean

• 80% (4/5) of samples need to be acceptable for PT to pass

• Even if there is only one unacceptable sample for PT, this must be investigated & corrected