Week 1 Introduction to Method Validation and Verification

Introduction to Method Validation and Verification

  • Welcome and introduction to the topic of method validation and verification in laboratory settings.

Analogy with Formula One Racing

  • Imagery of a Formula One driver:

    • Consideration of performance and selection of cars as analogous to laboratory assays.

    • The purpose of selecting a car: to win the championship.

  • Decision-making in car choice:

    • Simple choice presented initially; complexities arise when assessing specific attributes.

    • Key attributes to evaluate: performance on the track (analogous to assay performance).

  • Assays vs Cars:

    • Like cars, assays may differ in design and principle, but ultimately perform the same essential function (e.g., measuring glucose levels in serum samples).

    • Different assays (like cars) may perform better under distinct conditions or populations (e.g., pediatric patients).

  • Importance of Safety:

    • In both contexts, safety is paramount—driver and spectator safety in F1; patient safety in lab results.

Purpose of Method Validation and Verification

  • Gaining Confidence:

    • Validation is essential to ascertain that a method meets performance criteria, confirming its fitness for purpose.

  • Individual Laboratory Standards:

    • Criteria defining fitness can vary between laboratories, affecting the degree of validation required.

  • Requirements from Regulatory Authorities:

    • NADA (National Association of Testing Authorities) and TGA (Therapeutic Goods Administration) provide regulations requiring comprehensive validation of methods, especially for in-house or modified methods.

  • Understanding Error:

    • Validating methods provides a baseline for troubleshooting, necessary for clinical interpretation of results.

Method Validation Definitions

  • Validation Studies:

    • Definition: Provide objective evidence that a modified or in-house method is fit for its intended use.

    • Implication: If a kit is for research only, it becomes an in-house method requiring full validation.

  • Verification Studies:

    • Less extensive than validation but still important. They demonstrate a user’s ability to achieve published performance characteristics under their lab conditions.

Steps in Method Verification Process

  1. Initial Familiarization:

    • Understand basic principles and analytical requirements, especially if it’s a manual method.

  2. Quality Control (QC) Procedures:

    • Determine necessary QC levels:

      • Chemistry Assays: Typically two levels of QC are sufficient.

      • Immunoassays: Generally require three levels of QC with clinically relevant concentrations and normal ranges.

    • Define frequency of QC runs (e.g., twice a day, every eight hours).

  3. Quantification of Performance Aspects:

    • Analytical Range or Linearity assessment

    • Assay Imprecision analysis (random error measurement)

    • Assay Bias and Inaccuracy assessment (systematic error evaluation)

    • Analytical Specificity analysis (testing for known interferences)

  4. Method Acceptance:

    • Based on comparisons of observed vs. claimed performance, decide to accept or reject the method.

  5. Reference Interval Review:

    • Determine if a new reference interval is necessary for the implemented assay.

Evaluation Criteria in Method Validation

  • Minimum Performance Specifications:

    • Based on laboratory clinical needs and proficiency testing requirements.

    • Published Data Examples: RCPA (Royal College of Pathologists of Australasia), QAP (Quality Assurance Program), Westgard specifications.

  • Example: Troponin must have a coefficient of variation (CV) less than 10% at the 99th percentile of the normal population.

Documentation Standards

  • CLSI (Clinical Laboratory Standards Institute):

    • Provides guidelines within healthcare for performance measurements and method assessments.

  • Australian Standards:

    • NATA and NPAC provide guidance for method verification in Australian laboratories.

Key Components of Method Verification

Linearity

  • Definition: Assessment of the analytical range of the method through calibration standards testing.

  • Procedure:

    • Test six or more standards, including a blank.

    • Evenly spaced concentrations to ensure variance across the range.

    • Ideally, prepare standards independently to avoid errors from master solutions.

    • Calibration standards should be analyzed in duplicate or triplicate.

    • Data evaluation through scatter plots and regression analysis to establish linearity.

Residual Analysis

  • Definition: A residual represents the difference between observed values and predicted values.

  • Graphical Representation: Residuals plotted against concentration help visualize linearity appropriateness. Random distribution suggests linear regression suitability.

  • Patient Linearity Confirmation:

    • Assess linearity using patient samples to ensure matrix matching.

  • Analytical Range Specification: Ideally covering ranges from 0% to 150% of a clinically relevant concentration.

Limit of Detection and Quantification

  • Limit of Blank: Determined by analyzing 20 replicate samples of a blank solution, adding two or three standard deviations for assessment.

  • Limit of Detection: Lowest quantitatively detectable concentration, established via a similar method.

  • Limit of Quantification: Minimum concentration that can be measured at a stated CV percentage.

  • Examples of Functional Sensitivity: Troponin I < 10%, TSH < 20%.

Imprecision Method Assessment

  • Definition: Measuring the degree of random error inherent in the assay method.

  • Measurement Variables: Assays should use matrix-matched samples at medical decision points. Quality control samples are preferred for estimating imprecision.

  • Data Acquisition: Conduct multiple measurements over several runs or days to ensure robustness of data.

Method Bias and Accuracy Assessment

  • Definition: Measures the agreement between measured and true values, referred to as assay bias.

  • Assessment Procedures:

    • Analyze certified reference materials or compare results with gold standard/reference methods using patient samples.

  • Interpreting Comparisons: Attention to existing biases is critical for accuracy interpretation.

Specificity Assessment

  • Definition: Evaluating interference sources impacting assay results from both endogenous (internal) and exogenous (external) sources.

  • Endogenous Interference Examples: Hemolysis, icterus, lipemia; can create inaccuracies detrimental to clinical decisions.

  • Testing for Interferences: Prepare samples with added interferences while maintaining original specimen concentrations.

Clinical Scenario: Albumin Assay Verification

  • Background: Concern over the performance of the current albumin assay method and the introduction of a new BCP assay.

  • Significance of Albumin:

    • Albumin's structural properties and physiological roles.

    • Important for assessing liver function, nutritional status, risk of acute kidney injury, and as a prognostic factor in various conditions.

  • Challenges with Albumin Assays:

    • Historical context of BCG and BCP assays, including specificity and accuracy challenges with patient populations (especially hemodialysis patients).

Conclusion

  • Importance of method validation and verification in ensuring high-quality lab testing to improve patient outcomes.

  • Emphasis on continual evaluation and improvement of assays to meet clinical needs and regulatory standards.