Bias

Internal validity

• Do the observed results accurately reflect the true association?

• If a study lacks internal validity, external validity is irrelevant

• We do not compromise internal validity in an effort to achieve external validity

External validity (generalizability)

• To whom can the results be applied?

• Requires internal validity

Sources of error in epidemiologic research

• Random error

• Systematic error

• It’s possible to have both random and systematic error

Types of systematic error

• Selection bias

• Information bias

• Confounding

Random error

Error due to chance

Systematic error

Error due to recognizable source

Bias

Systematic error in design or conduct of study that leads to incorrect estimate of association

Sources of bias

• Can be caused by investigator or study participants during design or conduct of study

• Can occur in experimental, cohort, case-control, cross-sectional, and ecological studies

• Not inherent characteristic of study population

• Few studies have no bias or errors

Effects of bias

• Create appearance of an association when there is none, or mask and association that really exists

• Selection bias and information bias cannot be fixed in the analysis; confounding can be fixed to a point

Solutions to bias

• Limit in study design

• Limit in study conduct (during data collection)

• Critically evaluate after study has been completed, and discuss its effects

Evaluating magnitude of bias

• A small amount of bias does not usually have a major effect on the study results

• A large amount of bias can completely alter the study findings

Selection bias (in 2 × 2 table terms)

Who is in the 2 ×2 table

Information bias (in 2 × 2 table terms)

Where in the 2 × 2 table?

Selection bias

• Bias that can occur if people who agree to take part in a study are different from the source population you want to study

• Section bias has to do with who gets into your study

• Key: Participant differs on exposure and disease

How does selection bias occur?

Results from procedures used to select subjects into a study:

• At the time of recruitment

• During the process of retaining subjects in a study

Outcome of selection bias

Leads to an observed association that differs from what would have been obtained from source population targeted for study

When does selection bias occur?

• Case-control, cohort, and experimental studies

  • Case-control: Selection/participation of cases and controls is related to exposure status

  • Cohort/experimental: Selection/participation of exposed and unexposed subjects is related to disease status

• More likely to occur in case-control or retrospective cohort studies because exposures and outcomes have already occurred by the time a subject is selected into a study

Types of selection bias

• Inappropriate control selection/control-selection bias (case-control)

• Differential participation (case-control, cohort)

• Differential loss to follow up (cohort, experimental)

Control-selection bias

• Bias that can occur if controls are more (or less) likely to be selected if they are exposed (or unexposed)

• Occurs when controls fail to represent the exposure distribution in the source population from which the cases arose because controls do not accurately represent the same source population as the cases

How to prevent control-selection bias

Use identical selection criteria for cases and controls (recall purpose of controls)

Cohort studies of “dietary and lifestyle” factors often report lower mortality rates among participants compared to general population. Why?

• “Healthy participant effect”

• People interested in healthy lifestyles are more likely to take part in such a study than those with less healthy lifestyles

  • Have healthier behaviors

Differential participation

Bias that occurs if willingness or ability to participate is related to both exposure and disease status

How to prevent differential participation bias

Obtain high participation rates for all groups

Differential loss to follow-up

Bias that can occur if study participants exit a study for reason related to both exposure and disease

Why do we care about losses to follow-up?

If people are lost to follow-up, we cannot know whether they developed disease

How to prevent differential loss to follow-up

Since outcome cannot be known without good follow-up, must maintain high participation rates

Reducing selection bias

• Little (or nothing) can be done to fix selection bias once it has occurred

• Selection bias must be avoided through careful study design and conduct (see strategies for individual biases)

• Selection bias cannot be “controlled” in the analysis

How to assess selection bias

1. Describe possible sources

2. Describe likely impact on results (direction and magnitude of this bias)

3. Describe possible solutions

Information bias

• Bias that can occur if the information you collect from or about study participants is erroneous

• Has to do with the information that gets into your study

How does information bias occur?

• Occurs after participants have entered a study

• Results from differences in the way information is obtained

  • Study participants are incorrectly classified as exposed or unexposed, or diseased or not diseased

Outcome of information bias

Leads to an observed association that differs from what would have been obtained if all study participants had been classified correctly

When does information bias occur?

• More likely to occur in case-control or retrospective cohort studies because exposures (case-control) and outcomes (retrospective cohort study) have already occurred by the time a subject is selected into a study

• Case-control studies: Different techniques are used to collect information from cases and controls

• Cohort study: Different procedures are used to collect information from exposed and unexposed groups

• Bias will always occur when information differs on exposure and disease

Types of information bias

• Recall bias

• Interviewer bias

• Measurement error

Recall bias

• Bias that can occur if people with disease remember/report their exposure differently (e.g., more or less often) than people without disease

• Case-control study: Cases are more or less likely to recall prior exposures than controls

• Retrospective cohort study: Exposed participants are more or less likely to recall prior disease than unexposed participants

Solutions to recall bias

• Use controls who are also sick to promote comparable recall

• Use standardized, closed-ended questions to promote consistency and specificity

• Examine pre-existing data or use biological measurement to ascertain exposure

Interviewer bias

• Bias that can occur if there is a systematic difference in soliciting, recording, or interpreting data

• Case control study: Interviewer is influenced by participant’s case or control status

• Cohort and experimental studies: Interviewer is influenced by participant’s treatment or exposure status

Solutions to interviewer bias

• Use blinding/masking to prevent interviewers (or people collecting information about outcomes) to study hypothesis or to knowing whether someone is exposed

• Use high quality, standardized, closed-ended questionnaires to promote consistency and specificity, and use of appropriate response categories

• Examine pre-existing data

• Provide adequate and rigorous training for interviewers

Measurement/misclassification error

• Bias that can occur if study participants are placed in the wrong exposure or disease category

• Most common form of bias; present in all types of studies

Sources of measurement/misclassification error

• Self-reports

• Errors on medical records, death certificates, etc.

• Errors in how data are captured/data entry

• Non-specific disease or exposure definitions

Poor recall vs. recall bias

• Poor recall happens all of the time

• Recall bias occurs when recall is different for people who do and do not have the disease

Solutions to measurement/misclassification errors

• Improve accuracy of collected information

  • Use most accurate source of information available

  • Multiple measurements of exposure and disease

  • Validation: Corroborate the data using several sources

Reducing information bias

• Little (or nothing) can be done to fix information bias once it has occurred

• Information bias must be avoided through careful study designed and conduct

• Information bias cannot be “controlled” in the analysis

How to assess information bias

1. Describe possible sources

2. Describe likely impact on results (direction and magnitude of this bias)

3. Describe possible solutions