S

Week 5 Lecture 2 Notes – Errors, Bias & QA in Fingerprint Analysis

Errors and Bias

  • Error – Definitions
    • "A mistake" – e.g., spelling mistakes
    • A state/condition of being wrong in conduct or judgment – e.g., “the crash was caused by human error”
    • A deviation from accuracy or correctness – e.g., “his speech contained several factual errors”
  • Bias in Fingerprint Comparison
    • Concerned chiefly with cognitive, confirmation, and contextual bias
    • Cognitive bias – Influence of perceptual/mental processes on reliability & validity of observations/conclusions
    • Confirmation bias – Tendency to seek or interpret information that supports pre-conceptions
    • Contextual bias – Outside information or influences affect evaluation/interpretation of data

Quality Assurance (QA) & Quality Control (QC)

  • Purpose of Managing Quality – Ensures products/services meet or exceed customer expectations in both private & public sectors ("You get what you expected" – cheap vs. expensive car)
  • QA Characteristics
    • Prevents future issues by improving processes
    • Ongoing, circular (not linear)
    • Closely tied to training & adherence to Standard Operating Guidelines (SOGs)
    • Trained staff + clear SOGs ⇒ higher confidence & quality
  • QC Characteristics
    • Detects & corrects defects before release
    • Focuses on inspection & testing of final product/service
  • QA vs. QC
    • QA = maintaining a level of quality, preventive, active during development & beyond
    • QC = procedures to ensure product matches criteria, attempts to identify defects before dissemination
  • QA Example – Samsung Note 7 battery defect led to fires; recall cost Samsung 3.1 billion
  • QC Examples
    • Basic test on every apple-juice bottle; detailed tests on batch samples
    • High-speed train maker tests every component
    • Secret shoppers for retail service
    • Airline food samples verified for safety (not taste)

Maintaining QA & QC

  • Regular audits & reviews
  • Training & proficiency tests
  • Corrective actions & continual improvement loops
  • Documentation & traceability

Standard Operating Guidelines (SOGs/SOPs)

  • Written, organization-specific procedures that sustain QA & QC
  • Key Attributes
    • Concise, step-by-step, easy-to-read (KISS principle)
    • Sufficient detail for a minimally experienced user to succeed
    • Reviewed by \ge 2 trained/experienced individuals
    • Systematic periodic review (every 1–2 years)
    • Numbering system & master list
    • Current copies accessible in work areas
    • Multiple acceptable internal formats; must comply with regulations (health & safety, etc.)
  • Example Section – Marking Friction Ridge Impressions
    • Circle impression if item won’t be damaged & no nearby ridges
    • Place scale next to impression
    • Label with "R", number, date, examiner initials; add arrow for direction if needed
    • Photograph per photography protocol

Quality Assurance Programs – Fingerprint Comparison

  • Purposes
    • Ensure examiner competency
    • Ensure compliance with standards (qualifications, reporting, etc.)
  • Monitors & Tracks inconsistencies, admin errors, erroneous conclusions
  • Issues Addressed (non-exhaustive)
    • Training (basic & ongoing)
    • Evidence handling/storage
    • Health & safety (chemical hazards)
    • Examination & verification procedures
    • Conflict resolution
    • Administrative & testimony review
    • Corrective actions
    • Proficiency testing
    • Facility requirements
    • Accreditation/certification

Case Study: Shirley McKie (Scotland, 1997)

  • Background
    • DC Shirley McKie accused of leaving thumbprint (mark Y7) at Marion Ross murder scene in Kilmarnock (body discovered 6 Jan 1997)
    • 428 fingerprints recovered; Y7 (door-frame) attributed to McKie by 4 SCRO experts; gift-tag print XF to David Asbury (handyman, eventual defendant)
    • McKie denied entering beyond porch; prosecuted for perjury; acquitted 1999
  • Aftermath
    • Foreign experts consulted; 171 experts from 18 countries said Y7 not McKie’s
    • McKie awarded £750{,}000 compensation (2006)
    • Sparked Fingerprint Inquiry (2009–2011, 790-page report, 86 recommendations)
  • Open Questions
    • How did 4 examiners err?
    • Role of bias, inadequate methodology, verification failures
    • Prevention strategies pre-arrest

Fingerprint Inquiry (Scotland) – Selected Recommendations

  • Subjective Nature
    • Recognise fingerprint evidence as opinion, not fact (Rec 1)
    • Train examiners to emphasise personal-opinion basis (Rec 2)
    • Discontinue claims of 100\% certainty or infallibility (Rec 3)
    • Avoid term "dispute" for opinion differences (Rec 4)
  • Methodology & Contextual Bias
    • Reduce contextual bias; limit & record information given to examiners (Rec 6–9)
    • Features relied upon should be demonstrable to laypersons
  • Verification Process
    • Verifiers must be experienced, specially trained, & independent (Rec 29)
    • Blind verification: verifiers not told initial reasoning or shown marked images (Recs 30–32)

Case Study: Brandon Mayfield & Madrid Train Bombing (Spain/USA, 2004)

  • Incident
    • 11 Mar 2004 bombings killed 200 & injured 1400
    • Fingerprints on plastic bag of detonators (van near Alcalá station) circulated via INTERPOL
  • FBI Involvement
    • IAFIS produced 20 candidates; 19 Mar 2004 examiner identified Latent Fingerprint 17 (LFP 17) to Portland lawyer Brandon Mayfield
    • Second examiner verified; unit supervisor reviewed; 24-hour surveillance began
    • Spanish National Police (SNP) disagreed (13 Apr)
    • FBI sent examiner to Spain (21 Apr)
    • Mayfield arrested 6 May; independent expert (court-appointed) still agreed with FBI 17 May
    • Same day SNP identified print to Algerian Daoud Ouhnane; Mayfield released to home detention 20 May; FBI withdrew identification 24 May
    • Civil suit awarded Mayfield \$2 million

Causes of Error (FBI Panel Findings)

  • Incomplete analysis of LFP 17
  • Overconfidence in IAFIS automated search
  • High-profile case pressure (contextual bias)
  • Verification biased by prior conclusion (“tainted”)
  • Other Points
    • Claimed “unusual similarity” between Mayfield & LFP 17
    • Circular reasoning – interpreting latent features by working backward from known print
    • Over-reliance on Level 3 detail despite insufficient clarity
    • Rationalising away conflicting features
    • Failure to reassess after Spanish disagreement
    • Deficiencies in FBI verification procedure

Blind Verification in Fingerprint Workflows

  • Comparison packages assembled by 3rd party; identifiers removed
  • Contents: latent + 4–5 sets of known prints
  • Primary examiner unaware of context (AFIS hit vs. QC vs. suspect)
  • Second examiner (verifier) independent; package returned to 3rd party for results collation
  • Pros
    • Mitigates bias
    • More robust comparisons
    • Errors caught earlier
  • Cons
    • Longer turnaround
    • Increased examiner workload & pressure
    • Requires conflict-resolution mechanism

Real-World Bias Scenario

  • Crime-scene officer collected exhibits → lab (CFS)
  • DNA typing linked accused to scene
  • Later fingerprint processing on same exhibits yielded print identified to same accused
  • Defence alleged contextual bias: fingerprint examiner knew DNA result
  • Discussion points: amount of contextual info, independence, necessity of blind verification

Ethical & Practical Implications

  • Importance of acknowledging subjectivity; avoid absolutes ("100\% certain")
  • Training must include bias awareness & mitigation strategies
  • Systematic QA/QC, SOG adherence, & periodic review reduce error probability
  • High-profile or time-pressured cases heighten risk; blind verification and information isolation help

Connections & Takeaways

  • Both McKie & Mayfield cases illustrate catastrophic consequences of bias & QA failure
  • Reinforces recommendations for structured methodologies, limited context, independent verification
  • Need for organisational culture that embraces error detection, transparency, and continual improvement
  • Emphasis on human factors: cognition, pressure, confidence, and teamwork

Final Questions for Review

  • How can agencies balance efficiency with blind verification rigor?
  • What safeguards ensure SOGs stay current & followed in daily practice?
  • In what ways can technology (e.g., AI, automated ridge analysis) both reduce and introduce new biases?