Week 5 Lecture 2 Notes – Errors, Bias & QA in Fingerprint Analysis

Errors and Bias

• Error – Definitions
  • "A mistake" – e.g., spelling mistakes
  • A state/condition of being wrong in conduct or judgment – e.g., “the crash was caused by human error”
  • A deviation from accuracy or correctness – e.g., “his speech contained several factual errors”
• Bias in Fingerprint Comparison
  • Concerned chiefly with cognitive, confirmation, and contextual bias
  • Cognitive bias – Influence of perceptual/mental processes on reliability & validity of observations/conclusions
  • Confirmation bias – Tendency to seek or interpret information that supports pre-conceptions
  • Contextual bias – Outside information or influences affect evaluation/interpretation of data

Quality Assurance (QA) & Quality Control (QC)

• Purpose of Managing Quality – Ensures products/services meet or exceed customer expectations in both private & public sectors ("You get what you expected" – cheap vs. expensive car)
• QA Characteristics
  • Prevents future issues by improving processes
  • Ongoing, circular (not linear)
  • Closely tied to training & adherence to Standard Operating Guidelines (SOGs)
  • Trained staff + clear SOGs ⇒ higher confidence & quality
• QC Characteristics
  • Detects & corrects defects before release
  • Focuses on inspection & testing of final product/service
• QA vs. QC
  • QA = maintaining a level of quality, preventive, active during development & beyond
  • QC = procedures to ensure product matches criteria, attempts to identify defects before dissemination
• QA Example – Samsung Note $7$ battery defect led to fires; recall cost Samsung $3.1$ billion
• QC Examples
  • Basic test on every apple-juice bottle; detailed tests on batch samples
  • High-speed train maker tests every component
  • Secret shoppers for retail service
  • Airline food samples verified for safety (not taste)

Maintaining QA & QC

• Regular audits & reviews
• Training & proficiency tests
• Corrective actions & continual improvement loops
• Documentation & traceability

Standard Operating Guidelines (SOGs/SOPs)

• Written, organization-specific procedures that sustain QA & QC
• Key Attributes
  • Concise, step-by-step, easy-to-read (KISS principle)
  • Sufficient detail for a minimally experienced user to succeed
  • Reviewed by $\ge 2$ trained/experienced individuals
  • Systematic periodic review (every $1$–$2$ years)
  • Numbering system & master list
  • Current copies accessible in work areas
  • Multiple acceptable internal formats; must comply with regulations (health & safety, etc.)
• Example Section – Marking Friction Ridge Impressions
  • Circle impression if item won’t be damaged & no nearby ridges
  • Place scale next to impression
  • Label with "R", number, date, examiner initials; add arrow for direction if needed
  • Photograph per photography protocol

Quality Assurance Programs – Fingerprint Comparison

• Purposes
  • Ensure examiner competency
  • Ensure compliance with standards (qualifications, reporting, etc.)
• Monitors & Tracks inconsistencies, admin errors, erroneous conclusions
• Issues Addressed (non-exhaustive)
  • Training (basic & ongoing)
  • Evidence handling/storage
  • Health & safety (chemical hazards)
  • Examination & verification procedures
  • Conflict resolution
  • Administrative & testimony review
  • Corrective actions
  • Proficiency testing
  • Facility requirements
  • Accreditation/certification

Case Study: Shirley McKie (Scotland, $1997$)

• Background
  • DC Shirley McKie accused of leaving thumbprint (mark Y$7$) at Marion Ross murder scene in Kilmarnock (body discovered $6$ Jan $1997$)
  • $428$ fingerprints recovered; Y$7$ (door-frame) attributed to McKie by $4$ SCRO experts; gift-tag print XF to David Asbury (handyman, eventual defendant)
  • McKie denied entering beyond porch; prosecuted for perjury; acquitted $1999$
• Aftermath
  • Foreign experts consulted; $171$ experts from $18$ countries said Y$7$ not McKie’s
  • McKie awarded £$750{,}000$ compensation ($2006$)
  • Sparked Fingerprint Inquiry ($2009$–$2011$, $790$-page report, $86$ recommendations)
• Open Questions
  • How did $4$ examiners err?
  • Role of bias, inadequate methodology, verification failures
  • Prevention strategies pre-arrest

Fingerprint Inquiry (Scotland) – Selected Recommendations

• Subjective Nature
  • Recognise fingerprint evidence as opinion, not fact (Rec $1$)
  • Train examiners to emphasise personal-opinion basis (Rec $2$)
  • Discontinue claims of $100\%$ certainty or infallibility (Rec $3$)
  • Avoid term "dispute" for opinion differences (Rec $4$)
• Methodology & Contextual Bias
  • Reduce contextual bias; limit & record information given to examiners (Rec $6$–$9$)
  • Features relied upon should be demonstrable to laypersons
• Verification Process
  • Verifiers must be experienced, specially trained, & independent (Rec $29$)
  • Blind verification: verifiers not told initial reasoning or shown marked images (Recs $30$–$32$)

Case Study: Brandon Mayfield & Madrid Train Bombing (Spain/USA, $2004$)

• Incident
  • $11$ Mar $2004$ bombings killed $200$ & injured $1400$
  • Fingerprints on plastic bag of detonators (van near Alcalá station) circulated via INTERPOL
• FBI Involvement
  • IAFIS produced $20$ candidates; $19$ Mar $2004$ examiner identified Latent Fingerprint $17$ (LFP $17$) to Portland lawyer Brandon Mayfield
  • Second examiner verified; unit supervisor reviewed; $24$-hour surveillance began
  • Spanish National Police (SNP) disagreed ($13$ Apr)
  • FBI sent examiner to Spain ($21$ Apr)
  • Mayfield arrested $6$ May; independent expert (court-appointed) still agreed with FBI $17$ May
  • Same day SNP identified print to Algerian Daoud Ouhnane; Mayfield released to home detention $20$ May; FBI withdrew identification $24$ May
  • Civil suit awarded Mayfield $\$2$ million

Causes of Error (FBI Panel Findings)

• Incomplete analysis of LFP $17$
• Overconfidence in IAFIS automated search
• High-profile case pressure (contextual bias)
• Verification biased by prior conclusion (“tainted”)
• Other Points
  • Claimed “unusual similarity” between Mayfield & LFP $17$
  • Circular reasoning – interpreting latent features by working backward from known print
  • Over-reliance on Level $3$ detail despite insufficient clarity
  • Rationalising away conflicting features
  • Failure to reassess after Spanish disagreement
  • Deficiencies in FBI verification procedure

Blind Verification in Fingerprint Workflows

• Comparison packages assembled by 3rd party; identifiers removed
• Contents: latent + $4$–$5$ sets of known prints
• Primary examiner unaware of context (AFIS hit vs. QC vs. suspect)
• Second examiner (verifier) independent; package returned to 3rd party for results collation
• Pros
  • Mitigates bias
  • More robust comparisons
  • Errors caught earlier
• Cons
  • Longer turnaround
  • Increased examiner workload & pressure
  • Requires conflict-resolution mechanism

Real-World Bias Scenario

• Crime-scene officer collected exhibits → lab (CFS)
• DNA typing linked accused to scene
• Later fingerprint processing on same exhibits yielded print identified to same accused
• Defence alleged contextual bias: fingerprint examiner knew DNA result
• Discussion points: amount of contextual info, independence, necessity of blind verification

Ethical & Practical Implications

• Importance of acknowledging subjectivity; avoid absolutes ("$100\%$ certain")
• Training must include bias awareness & mitigation strategies
• Systematic QA/QC, SOG adherence, & periodic review reduce error probability
• High-profile or time-pressured cases heighten risk; blind verification and information isolation help

Connections & Takeaways

• Both McKie & Mayfield cases illustrate catastrophic consequences of bias & QA failure
• Reinforces recommendations for structured methodologies, limited context, independent verification
• Need for organisational culture that embraces error detection, transparency, and continual improvement
• Emphasis on human factors: cognition, pressure, confidence, and teamwork

Final Questions for Review

• How can agencies balance efficiency with blind verification rigor?
• What safeguards ensure SOGs stay current & followed in daily practice?
• In what ways can technology (e.g., AI, automated ridge analysis) both reduce and introduce new biases?