Validation of analytical methods

FOODBORNE VIRUSES

Examples include norovirus, hepatitis A virus, hepatitis E virus, etc.

You can get these with oysters, mussels, strawberries, etc.

DETECTION OF VIRUSES IN FOOD AND WATER

Several steps are involved in this processes. Low levels of viruses in the primary concentration. So we concentrate them even more. We have a lower volume but we need to concentrate them even more so that we can extract the nucleic acid extraction.

These methods can be qualitative (+/-) or quantitative. They are performed in different manners.

LIFECYCLE OF AN ANALYTICAL METHOD

We need our tests to be reliable, as we don't want false negatives or false positives.

So we can design a new method and validate it, where we do a set of experiments to check if it's fit for purpose. Then we do an accreditation, where we do a formal recognition of confidence, and that it complies with standards.

Maybe a method has been validated in other labs, so all we need to do is a verification (small scale validation).

Quality is maintained by:

• Proper shipping and storage of samples

• Training of personnel

• Routines to avoid contaminations, etc.

• Performance controls

• Internal controls

• Proficiency testing

• Internal and external audit (check that everything is still valid), etc.

VALIDATION

This occurs when you determine a set of performance characteristics of an analytical method. It helps you decide whether your method is fit for its purpose. There are different performance characteristics:

HOW DO WE VALIDATE?

We have to consider a few things when planning a validation study:

1. Which performance characteristics do we need to assess? Determined by whether it's qualitative or quantitative. There might also be specific requirements from within the field.

2. Should we use artificially contaminated material or naturally contaminated material? We have to determine with samples are specifically needed for the method we want to do. Artificial contamination might not be very realistic for example.

3. Which sample types should we use?

4. How many samples do we need?

5. What concentration range should we validate for?

Overall, we need to be realistic and cost-efficient manner.

LIMIT OF DETECTION

This is the lowest concentration of analyte in a sample that can be detected with a stated probability. Often reported as LOD 50% or LOD 95% (probability of detection).

This is a dilution series with viral RNA from HAV. There are 10 dilution levels and 32 samples per dilution level. So you have your virus with a dilution series (high to low concentration). You measure how many samples are positive and the fraction at each dilution level. There is a decrease as some samples are very diluted.

It's a yes or no outcome.

ACCURACY

This is the level of agreement between a test result and an accepted reference value (ISO 5725). This is based on:

• Trueness

• Precision

  • Repeatability

  • Reproducibility

TRUENESS

This is the level of agreement between the average for a large series of test results and an accepted reference value.

There can be a systematic error. Maybe if you've developed a pipette that gives you one more microliter than it's supposed to. Often reported as a bias.

For each method you use the reference and alternative method. You do a regression line to see the average agreement to check if they perform equally.

You can also plot the mean and the bias. The bias is close to 1 which is good, meaning that there is not much bias.

PRECISION

This is the level of agreement between independent measurement results obtained under specified conditions.

There can be a random error. Often reported as a standard deviation or a coefficient of variation. This can be due to a person who's not trained enough for a specific method, so they make mistakes. It's more due to the process.

PRECISION (REPEATABILITY)

Repeatability conditions: when independent measurement results are obtained using identical sample material, within the same lab, with the same equipment and within a short period of time.

This is the repeatability testing of a standardised method for analysis of norovirus GII in oysters. You do 9 different repetitions, all the same, and you see that there is some fluctuation. There is a high random error.

PRECISION (REPRODUCIBILITY)

Reproducibility conditions: when measurement results are obtained with the same method for identical sample material, at different laboratories, with different operators and with different equipment.

This is between-lab testing of a standardised method for analysis of norovirus GII in oysters. There are 23 labs. There is an even larger error.

ACCURACY: SUMMARY

Trueness and precision are not exactly related to each other.

SENSITIVITY

This is the probability of a positive test result, given that the tested item is a true positive.

Sensitivity = A/(A+C)

Sensitivity = 95/(95+5) = 0.95

We have patients who we know have the disease. So we test the method with them. 5 of those are negative so we know it's a false negative.

SPECIFICITY

The probability of a negative test result, given that the tested item is a true negative.

Specificity = D/(B+D)

Specificity = 180/(20+180) = 0.9

This is the same case as before but with negative results.

POSITIVE PREDICTIVE VALUE

How should we interpret a positive test result?

PPV: the probability that the tested item is a true positive, given a positive test result.

PPV = A/(A+B)

PPV = 95/(95+20) = 0.8261

Test is positive is what we knew from beforehand.

NEGATIVE PREDICTIVE VALUE

This is the probability that the tested item is a true negative, given a negative test result.

NPV = D/(C+D)

NPV = 180/(5+180) = 0.973

PPV, NPV AND PREVALENCE

PPV and NPV depend not only on the sensitivity and specificity of the analytical method, but also on prevalence.

The probability that the patient actually has the disease is 33%. But on the other hand, a negative test is very reliable. You need to improve the specificity.

This is a disease with a higher prevalence. The performance will improve if the disease is more common. Useful if you test a group of people with specific symptoms. Not useful for a screening test.

HOW TO CALCULATE PPV AND NPV FOR A DIAGNOSTIC TEST?

PPV AND NPV AT DIFFERENT PREVALENCE

SUMMARY

Validation: when you determine a set of performance parameters to point out the strengths and limitations of an analytical method. Helps you to decide whether the method can be used for its intended purpose.