Bias & Human Factors - Comprehensive Notes

Human Factors Topics

• Cognitive Bias in Forensic Science
  • Types and importance of cognitive bias and its relevance to forensic science.
  • Factors that can introduce cognitive bias into a system.
  • Difference between task-relevant and task-irrelevant information.
  • Methods a laboratory may use to safeguard against cognitive bias.
  • How bias may impact experts when communicating in and out of court.
• Terminology
  • base-rate expectations
  • task-relevant
  • task-irrelevant
  • blinding
  • suspect-driven bias
  • adversarial system
  • inappropriate influence
  • linear sequential unmasking

Human Factors Terminology

• Confidence Hardening
• Cognitive Dissonance
• Base Rate Expectations
• Expectation Effects
• Role Effects
• Linear Sequential Unmasking
• Reconstructive Effects
• Focalism
• Anchoring Effects
• Expectation Bias
• Cognitive Bias
• Contextual Bias
• Motivational Bias
• Confirmation Bias

What are Human Factors?

• Human factors encompass psychological and physiological traits, behaviors, and limitations affecting human interactions with systems and tasks.
• In forensic science, human factors significantly influence the accuracy and reliability of analyses and judgments.
• Key human factors include cognitive processes, decision-making, expertise, social and environmental influences, and physical factors.
• Understanding human factors is crucial for designing systems to reduce errors and biases in forensic work.
• Strategies for limiting errors, bias, and fostering employee awareness and wellness are explored to create a less biased environment

Bias and Human Factors Categories

• Categories of bias
• Base-rate expectations/expectancy effects
• Task relevance
• Six fallacies of bias
• Eight sources of bias
• Minimizing bias
• Risk mitigation
• Examiner well-being

Bias Definition

• "A particular tendency or inclination, especially one that prevents unprejudiced consideration of a question"

Cognitive Biases

• cognitive bias
• implicit bias
• cognitive dissonance
• tunnel vision
• confirmation bias
• interpretive bias
• belief perseverance
• asymmetrical skepticism

Cognition

• Cognition involves awareness, perception, reasoning, and judgment
• Cognitive processes include mental shortcuts intended to speed up decision-making
• Cognitive bias may occur when these shortcuts carry incorrect inferences
• Subconscious type of bias for which a forensic examiner may not be aware
• Cognitive bias can also affect our memory and reasoning processes
• “We do not experience the world as it really is….”

Human Memory

• Human Memory is Unreliable
• Memory fails without awareness
• Many factors affect memory
• Encoding, storage, and retrieval are complex processes

Cognition and Information Processing

• People do not passively receive and encode information.
• As information is received, it is processed.
  • Identify and make sense of it
  • Interpret and assign it meaning
  • Compare it to information already stored in memory
• Theoretical and conceptual frameworks exist for perception, judgment, and decision-making
• Our perception and judgment are influenced by a variety of cognitive processes that are not dominated by the actual data

Cognitive Bias and Information Interaction

• When cognitive biases exist, we interact differently and subjectively with the information
• During examination of the data, we are more likely to notice and focus on characteristics that validate and conform to extraneous information or context, a belief, or a hope
• Tend to avoid and ignore data that conflict and contradict such biases and disconfirm data that we notice are ignored
• The way we search and allocate attention to the data can be selective and biased
• These and other manifestations of bias and cognitive influences can make perception, judgment, and decision-making unreliable

Contextual Information and Decisions

• Mood, prior experiences, and peripheral information can help decision-making or lead to confirmation bias.
• Being exposed to extra information may influence the analytical process.

Expertise

• Expertise is domain and task-specific.
• We have limited insight into how we make decisions.
• Experience does not necessarily translate to expertise and cannot avoid human factors.
• Base rate expectations of an expert can often affect expert decision-making.

Experience vs Expertise

• Experience does not necessarily translate into expertise.
• Experts must demonstrate superior accuracy relative to novices.
• Experience alone is an insufficient predictor of expertise and performance.
• Experience may bring about “expectations” that are not derived from the actual data i.e., “base rate expectations”

Confidence Hardening

• Confidence in a decision tends to increase over time
• Especially if the person who made the decision receives some sort of confirmation
• This is termed “confidence hardening”

Cognitive Biases List

• Expectation bias
• Confirmation bias
• Contextual bias
• Anchoring effects
• Role effects
• Motivational bias
• Reconstructive effects

Expectation Bias

• Also known as the observer expectancy effect
• Expectations or beliefs influence perceptions and interpretations of data
• When the expectation of what will be found, influences what is actually found
• Unconscious bias leads to skewed findings or judgments aligning with preconceived notions or desired outcomes
• Analysts may interpret evidence to confirm initial suspicions of guilt or innocence
• Analysts expecting a result may prioritize evidence supporting their expectations
• Contradictory evidence might be overlooked or misinterpreted

Confirmation Bias

• Tendency to favor information that supports preexisting beliefs or hypotheses
• More weight is given to confirming evidence, while contradicting evidence is undervalued
• Looking for confirming evidence rather than potentially conflicting evidence
• Like expectation bias, confirmation bias emphasizes minimizing results that contradict expectations
• Analysts may focus on evidence supporting a suspect’s guilt
• Contradictory or exonerating evidence might be ignored or downplayed

Contextual Bias

• Influence of irrelevant information or external factors on data interpretation
• When external information influences the outcome
• Contextual bias can be conscious or unconscious
• Bias can occur without the analyst’s awareness
• Stems from expectations or decisions
• Can skew or produce incorrect conclusions in data analysis
• Analysts may unconsciously align findings with perceived expectations of investigators, attorneys, or the public

Anchoring Effects

• Cognitive bias where initial information (the “anchor”) overly influences decisions
• Sets a baseline that affects subsequent evaluations, even if irrelevant
• Relying too heavily on an initial piece of information when making subsequent judgments – also termed focalism
• Analysts may fixate on initial evidence, overlooking other relevant details
• Initial suspicion of a prime suspect can bias evidence interpretation
• Anchoring can lead to skewed conclusions based on early, potentially flawed, information

Role Effects

• Influence of professional or social roles on perceptions, decisions, and behaviors
• Expectations, responsibilities, and norms associated with a specific role
• Role-related biases can shape information interpretation and decision-making
• Analysts and investigators may experience biases affecting evidence analysis and conclusions
• Analysts might unconsciously align findings with prosecution expectations
• When the scientist identifies as part of the prosecution or defense team

Motivational Bias

• Bias driven by personal desires, interests, or goals
• Influences judgments and decisions, leading to skewed conclusions favoring desired outcomes
• Affects objectivity and accuracy of findings in investigations
• Analysts may unknowingly shape findings to support convictions or meet expectations
• Pressure to align results with the prosecution’s theory for potential promotion or recognition
• When results “line up” with a favored or desired conclusion

Reconstructive Effects

• Memory and perception are altered by new information, experiences, or context
• Leads to less accurate and more distorted recollections of past events
• Affects eyewitness testimony and the recollection of events by investigators and analysts
• Relying on memory instead of contemporaneous notes can introduce inaccuracies
• Analysts may fill memory gaps during testimony using current analysis rather than original details

Categories of Bias (Summary)

• Expectation bias
  • When the expectation of what will be found affects what is actually found
• Confirmation bias
  • Looking for confirming evidence rather than potentially conflicting evidence
• Contextual bias
  • When external information influences the outcome
• Anchoring Effects (or focalism)
  • Relying too heavily on an initial piece of information when making subsequent judgments
• Role effects
  • When the scientist identifies as part of the prosecution or defense team
• Motivational bias
  • When results line up with a favored or desired conclusion
• Reconstructive effects
  • When an analyst relies on memory (fills in the gaps) rather than notes

Factors Introducing Bias

• Attention, perception, and memory
  • Attention
    • Ability to focus on relevant information and ignore distractions; critical for forensic practitioners to remain unbiased and detail-oriented
  • Perception
    • Interpretation of sensory information, often influenced by expectations, which can introduce bias
  • Memory
    • Human memory is prone to errors, distortions, and alteration by circumstances like trauma or incomplete information

Heuristics

• Mental shortcuts or rules of thumb used to simplify complex decision-making and problem-solving
• Operates like an "auto-complete" in the brain, enabling quick judgments based on experience
• Saves time and mental effort but can lead to biases or errors under certain conditions
• Experts may use the representativeness heuristic to assess evidence similarity, such as fingerprints or DNA
• Overreliance on representativeness can lead to overestimating matches and potential errors in conclusions
• Balance heuristics with rigorous analysis and consideration of all available evidence

Factors Introducing Bias

• Lack of proper risk assessment
  • Risk assessment is the ability to evaluate the probability and consequences of various outcomes
  • Misjudgments in risk assessment can affect forensic conclusions
• Training and experience
  • The level and quality of training received by forensic practitioners can influence the likelihood of errors and bias occurring; experience ≠ expertise
• Workplace dynamics
  • Workplace dynamics and team relationships can have a profound impact on bias
• Wellness, stress, and fatigue
  • Stress and fatigue can significantly impair decision-making and increase the likelihood of errors and biases

Dror (2020) - Categories of Bias

Category A

*   Human Nature

Category B

*   Case Specific

Category C

*   Perspective

8 SOURCES OF BIAS

*   Data
*   Reference Materials
*   Contextual Information
*   Base Rate and Experience
*   Education and Training
*   Organizational Factors
*   Environment, Culture
*   Personal Factors

Categories of Bias - Dror (2020)

• Category 1: Case-specific details
  • Data related to a specific forensic analysis
  • Examples: item of evidence itself, reference materials used to conduct the analysis, and any contextual information that accompanies the case
• Category 2: Environmental, cultural, or experienced-based sources of bias
  • Related to the individual and their surroundings and not the details of the case
• Category 3: Human cognitive factors and processes within the brain
  • Human factors are a category of biases that occur regardless of the specific analysis or individual and are a result of the function of the human brain

Fallacies of Bias - Dror (2020)

Processes Susceptible to Bias

• DNA mixture interpretation
• Fingerprints
• Forensic Anthropology
• Blood patten analysis
• Dog handling
• Handwriting
• Document examination
• Shoeprints
• Toolmarks
• Crime Scene
• Forensic pathology

The Blind Spot

• The "Blind Spot" is Real

Beliefs about the Scope of Bias

Risk

• NONCONFORMANCES

Context of Evidence

• Context of Evidence is Important
  • Helps assess the strength of inferential connection between evidence and propositions

Task Irrelevant Information

• Forensic scientists are often influenced by task irrelevant information.
  • Suspect criminal history, confession, or alibi
• Cannot detect whether they are being influenced by bias.
• Information is task-relevant if it is necessary for drawing conclusions about the hypothesis in question.

Risk of Bias

• Higher risk of bias when the evidence being analyzed is of low quality
• Can affect the observation of data, testing strategies, how the results are interpreted, and how conclusions are reached
• Potential risks include a failure to express alternative explanations or a reluctance to express doubt

Juror Issues

• The expression of uncertainty or probability can influence juror perception
• Jurors may fail to consider alternative explanations unless they are made explicit
• They will search for evidence to support their working hypothesis

Confirmation Bias Reminder

• The tendency to confirm an initial theory or preconception and avoid disconfirming information
• Preexisting beliefs, expectations, motives, and situational context influence the collection, perception, and interpretation of evidence

Examiner Decision-Making Variables

• Influence of case information
• Influence of comparison procedure
• Influence of previous conclusions
• Emotional factors

Other Factors Influencing Cognitive Bias

• Between-person design
• Within-person design
• “Context-free” vs. context – Suspect confessed/in custody etc.
• Experience/training
• Time pressure
• Previous analyses/results/factors – (e.g., “exclusion case”, anonymous vs. “high profile” expert)
• Suspect details e.g., ethnicity
• Exemplar comparison – (i.e., targeted/single vs. multiple)
• Emotional context – (i.e., gravity of case, physical harm/violent crime, photographs etc.)
• Complexity or sample quality

Cognitive Bias: Problems and Solutions

• Case-specific information about the suspect of crime scenario
• Procedures for use of exemplars
• Knowledge of a previous decision/expectation
• Reduce access to unnecessary information
• Control the order of providing relevant information
• Restrict access to task-irrelevant information
• Health and well-being of examiners

CASE MANAGERS CAN MITIGATE ISSUES

Strategies to Mitigate Bias

• Linear Sequential Unmasking
• Blinding
• Structured approaches
• Feedback mechanisms
• Improving workplace culture
• Reducing Stress and fatigue

Mitigation Strategies: Independent Reviews

• Unless genuinely independent, review and verification might not be effective
• Group decision-making may introduce problems and biases
  • Transference of error
  • Seeking conformity
  • Deference, etc.
• Forensic practitioners should strive for independent reviews and explain the nature of the review process

Mitigation Strategies

• Acknowledge the bias exists
• Be aware of potential consequences
• Implement safeguards
  • Sequential unmasking
  • Context management
  • Case managers
  • Blinding precautions
• Awareness, training, and competency

Task Relevance & Linear Sequential Unmasking

• NCFS document
• Task relevance varies throughout the investigation
  • Crime scene collection > analytic phase > interpretation phase
• Contextual information may be necessary for some analytical phases
• Mitigate risk by eliminating task-irrelevant information
• Linear sequential unmasking (LSU)
  • Examine evidence in isolation from reference material
  • Limited ability to change their result after exposure to the reference material
  • Case managers
  • Document changes

Ensuring Task-Relevant Information

• Forensic scientists should rely solely on task-relevant information when performing forensic analyses
• OSAC should specify what types of information are task-relevant for common forensic tasks
• Forensic labs should take steps to avoid exposing analysts to task-irrelevant information through the use of context management procedures

Human Factors: Workplace Stress & Well-Being

Internal pressures:

• Dynamic environment
• Workload volume
• Tight deadlines
• Lack of advancement
• Working hours
• Low salary
• Technology distractions
• Fluctuating priorities

Industry-specific pressures:

• Technique criticism
• Exposure to distressing material
• Adversarial legal system
• Zero tolerance for “errors”

FSSPs and Workplace Wellness

• FSSPs must promote workplace wellness
• Mindfulness programs:
  • Decrease stress
  • Improve focus/efficiency

Readings

• Dror, I.E., et. al. (2018). Human Factors Effecting Forensic Decision Making: Workplace Stress and Well-being. Journal of Forensic Science, 63(1), 258–261.
• Edmond, G., et al. (2017). Thinking forensics: Cognitive science for forensic practitioners. Science and Justice, 57, 144–154.
• Cooper, G. S. at al. (2019). Cognitive bias research in forensic science: A systematic review. Forensic Science International, 297, 35–46.
• National Institute of Standards and Technology. Ensuring That Forensic Analysis is Based Upon Task-Relevant Information.
• Dror, I. E. (2020). Cognitive and Human Factors in Expert Decision Making: Six Fallacies and the Eight Sources of Bias. Analytical Chemistry, 92, 7998–8004.
• Forensic Science Regulator Guidance. (2020). Cognitive Bias Effects Relevant to Forensic Science Examinations (in BB)
• Kukucka, J, et al. (2017) Cognitive Bias and Blindness: A Global Survey of Forensic Science Examiners. Journal of Applied Research in Memory and Cognition, 6, 452-459.
• TFSC video in Blackboard for viewing

Assignment

• Individual Submission Bias assignment:
• Write a concise one-page personal statement (no more than 500 words) regarding a personal bias that you have. Personal statements will not be distributed to anyone else.
• Provide an electronic copy via email (bns017@shsu.edu) by February 18th
• Bonus study material:
• View the TFSC human factors presentation here: (Audio) human factors by leigh tomlin on prezi next
• Use the arrow button at the bottom of the screen to hear each slide