quality assurance/quality control

quality assurance

• set of activities for ensuring quality in the processes by which products are developed

• process oriented

• focus on defect prevention

Quality control

• set of activities for ensuring quality in the actual product

• product oriented

• focus on defect identification and correction.

quality triangle: proficiency testing

Personnel: certification

laboratories: accreditation

standards: standardization

forensic specialties accreditation board, inc.

started in 2000 accreditation of the certifiers.

Quality assurance: who set the standards?

FBI/DEA/ATF/NIJ

review new technologies

reccomend changes in methods and procedures

improve discipline principles

develop consensus.

current heirarchy of standards for accrediting bodies to use in auditing U.S. forensic DNA laboraties

top to bottom

• international laboratory accreditation corperation (ILAC)

• something 17025:2005

• FBI quality assurance standards 2011

• SWGDAM guidelines

Standard operating procedures (SOP’S)

a set of written instructions that describes, in detail, how to perform a process safely and effectively.

why is an SOP important

• to ensure a test is performed correctly

• to ensure a test is performed consistently

• to ensure quality of test results

• to ensure adherence to regulatory requirements

• to ensure safety while performing a test.

elements of an SOP

• title

• scope

• purpose

• definitions

• roles and responsibilities

• materials

• procedures (process)

• references

• revision history

Method validation: when to validate?

before a new method is used

when conditions change

and to modify the exisiting method

key characteristics of method validation

• accuracy

• precision

• limit of detection

• limit of quantitation

• linearity

• range

• robustness

• ruggedness

• system suitability

• specificity

• selectivity/sensitivity

accuracy

the closeness of test results to the ‘true” value

precision

measure of reproducibility and consistency of test results over repeated trials.

limit of detection

the lowest concentration of an analyte that can be reliably detected but not necessarily quantified.

limit of quantitation

the lowestconcentration of an analyte that can be reliably quantified in a sample.

selectivity/ sensitivity

the ability of a test to accurately identify the presence of a specific analyte in the presence of other substances. Sensitivity refers to the test's ability to detect low concentrations of the analytes.

specificity

the ability of a test to correctly identify the absence of an analyte. It indicates how well the test distinguishes between the target analyte and other substances.

linearity

the ability of a test to reproduce results that are directly proportional to the concentration of an analyte in a sample

range

the concentration interval within which an analytical method can reliably produce accurate and precise results.

robustness

the measure of a test’s capacity to remain unaffected by small but deliberate variations in procedural parameters listed in the documentation

ruggedness

The degree of reproductibility of test results obtained by the analysis of the same samples under a variety of conditions

system suitability

A set of tests performed before analyzing samples to verify that the analytical system (including the instrument, column, and operating conditions) is functioning properly and within acceptable parameters, ensuring reliable results for the intended analysis.

error

difference between actual and observed measurement

random error

caused by unknown reasons

systematic error

caused by instrument or handler

uncertainty

-measure of the variability of total error. It is amount or % that an observed or calculated value differs from the actual.

Required by ISO 17025

Quality assurance: laboratory accreditation

laboratory accreditation demonstrates that its management, operations, personnel, procedures and instruments, physical plant and security, and personnel safety procedures meet certain standards

provides a public way to identify standards, and which laboratories are meeting them

independent, impartial, objective system of review

QUALITY ASSURANCE THROUGH INSPECTION

external inspection: alternate years

• facilities

• equipments

• written SOP’S

• review of case work report

• interview of technical staff

internal periodic case report and case note review

• examiners are following established SOP’s

• findings are properly documented

proficiency testing(internal and/or external)

blind test: true results are unknown to examiner

national academt of sciences report (2009)

• congress mandated report

• found serious deficiencies and called for major reforms

• overall, labs are underfunded and understaffeddisparity in labs in funding, access to instrumentation, skilled/well trained personnel, certification, accreditation, and oversight.

NAS reccomendations

Creation of national institute of forensic sciences (NFIS)

• lead research

• establish and enforce standards

• oversee education standards-FEPAC

independence from police/ prosecuotres office certification mandatory

• written exams

• supervised practice

• proficiency tests

• adherance to a code of ethics

NAS reccomendations part 2

Accreditation mandatory

establish QC procedures

• confirm validity (robustness, reproductibility, and reliability) of a procedures

• level of uncertainty

• identify mistakes, fraud, and bias

Courtroom Testimony

• discipline must be grounded in reliable scientific methodology

• must reveal level of human interpretation of results

• must clearly define limitations of analysis