Error and Bias
Comprehensive Study Notes: Bias and Error in Epidemiology
1. Fundamentals of Epidemiologic Error and Population Sampling
Epidemiologic research relies on assessing error as the essential first step toward establishing causal inferences. As articulated by statistician Dr. John C. Bailar, III of the National Academy of Sciences, the central question in evaluating study findings is not whether uncertainties exist within the data, but whether those uncertainties are so substantial that they preclude drawing valid conclusions. Evaluating an association requires systematically ruling out non-causal alternative explanations by identifying, minimizing, or eliminating both random and systematic errors.
Understanding study error requires examining the hierarchy of sampling through which study participants are selected from broader populations.
Population Hierarchy
Sampling establishes how individuals are selected from a larger group to represent the health phenomenon under study. Epidemiologic studies distinguish between four nested population levels:
General Population: The total overarching group or macro-level population.
Target Population: The specific subgroup of individuals to whom study findings are intended to apply. For example, in a study evaluating hormone replacement therapy (HRT) and breast cancer risk, the target population consists of all women with breast cancer residing in San Antonio.
Source Population: The subset of eligible subjects available for study recruitment, also known as the sampling frame. In the San Antonio HRT study, the source population comprises female breast cancer patients diagnosed within the University Health System, excluding non-San Antonio residents to maintain target population alignment.
Study Sample: The actual group of individuals selected from the sampling frame who participate in the research (for example, female breast cancer patients residing in San Antonio diagnosed within the University Health System).
Sampling Methodologies
The selection method directly determines the degree of error and generalizability of study results:
Random Sampling: Every member of the source population possesses an equal probability of selection. This approach requires a complete, enumerated sampling frame and provides the optimal method for minimizing selection-related error.
Stratified Random Sampling: The source population is partitioned into distinct strata based on specific demographic or clinical characteristics (such as race, ethnicity, or gender), followed by random selection within each stratum. Large national surveys utilize stratified sampling to guarantee proportional representation across demographic groups.
Convenience Sampling: Participants are selected based on accessibility (for example, recruiting individuals at specific community gathering points or colonias). This method is employed when source populations cannot be fully enumerated, such as transient or hard-to-reach populations (e.g., studying illegal drug use and HIV among undocumented Texas workers). Convenience samples carry higher vulnerability to error but prove useful for pilot testing study methods.
Snowball Sampling: Initial participants identify and refer subsequent participants who share relevant exposure or outcome characteristics. Like convenience sampling, snowball sampling is prone to selection errors but valuable for exploratory or small-scale research.
2. Taxonomy of Epidemiologic Errors: Random vs. Systematic
Epidemiologic errors fall into two distinct structural categories: random errors and non-random (systematic) errors.
Systematic Comparison of Error Types
The fundamental properties of random and systematic errors differ across statistical, methodological, and operational dimensions:
Dimension | Random Error | Systematic Error (Bias) |
Nature of Flaw | Non-systematic, chance fluctuations around the true parameter value. | Non-random, systematic flaw in design, conduct, or analysis. |
Primary Impact | Reduces precision and reliability of the study. | Compromises internal validity and accuracy of the study. |
Statistical Effect | Expands variability around effect estimates (e.g., wider 95\% Confidence Intervals). | Alters the calculated point estimate of effect (e.g., Odds Ratio OR or Relative Risk RR). |
Effect of Increasing Sample Size (n) | Decreases as study size increases, approaching zero at infinite sample size. | Unchanged by study size; persists regardless of sample scale. |
Subtypes of Random Error
Random error represents unpredictable variation surrounding the true population parameter and encompasses two primary forms:
Random Sampling Error: Occurs when a selected sample fails to accurately reflect the source population purely due to chance ("bad luck"). While unavoidable, random sampling error is reduced by expanding the study sample size.
Random Measurement Error: Arises from non-purposeful inaccuracies during data collection, such as ambiguous survey questions, laboratory measurement fluctuations, or random data entry errors. This results in non-differential misclassification and is minimized by improving instrument reliability and taking repeat measurements.
Definition of Systematic Error (Bias)
As defined by Gordis, bias constitutes:
"Any systematic error in the design, conduct, or analysis of a study that results in a mistaken estimate of an exposure's effect on the risk of disease."
Systematic error occurs when the average results of a hypothetically infinite number of study repetitions differ from the true population value. Bias can originate from investigators or participants at any study stage.
3. Directional Impact of Bias on Measures of Effect
Bias alters computed measures of association—such as the Relative Risk (RR) or Odds Ratio (OR)—relative to the null hypothesis value of 1.0 (representing no association).
Classification of Directional Shifts
The distortion caused by systematic error shifts the estimated association either toward or away from the null value:
Biased Toward the Null (Underestimation):
Positive Association (RR > 1.0 or OR > 1.0): The observed effect estimate is lower than the true effect, causing a harmful association to be underestimated.
Inverse/Protective Association (RR < 1.0 or OR < 1.0): The observed effect estimate is shifted upward toward 1.0, underestimating the true protective effect.
Biased Away from the Null (Overestimation):
Positive Association (RR > 1.0 or OR > 1.0): The observed effect estimate is higher than the true effect, causing a harmful association to be exaggerated.
Inverse/Protective Association (RR < 1.0 or OR < 1.0): The observed effect estimate is shifted further below 1.0, exaggerating the true protective effect.
4. Selection Bias: Mechanisms, Manifestations, and Prevention
Selection bias stems from systematic errors introduced during subject enrollment or retention, causing the relationship between exposure and disease to differ between study participants and non-participants.
Manifestations Across Study Designs
Selection bias distorts non-representative samples and manifests across different research designs:
1. General and Recruitment Biases
Healthy Volunteer Bias: Occurs when individuals who choose to participate in research possess systematically healthier behaviors or statuses than non-respondents.
2. Case-Control Study Selection Biases
Selection probabilities among cases and controls differ based on exposure status:
Berkson's Bias: Arises in hospital-based case-control studies when both the exposure and disease independently increase the likelihood of hospitalization, artificially inflating or distorting the exposure-disease association.
Inappropriate Control Exclusion: Occurs when controls are selected dependent on exposure. For example, in a study of alcohol consumption and liver cancer, excluding controls with alcohol-related digestive disorders creates a control group with unnaturally low alcohol consumption, distorting the Odds Ratio.
3. Cohort Study Selection Biases
Selection or retention probabilities within exposed or unexposed groups relate to the outcome risk:
Loss-to-Follow-Up (Attrition Bias): Occurs when participants who drop out differ systematically from those retained. For instance, in a smoking and lung cancer study, if high-risk smokers with early symptoms drop out or die from competing causes, the true risk ratio is distorted. Similarly, in a diet and cardiovascular disease study, attrition among participants with poorer diets and higher disease rates skews outcome measures.
Healthy Worker Effect: In occupational cohort studies, employed workers exhibit lower overall mortality/morbidity than the general population because severely ill or disabled individuals are excluded from employment.
Differential Non-Response & Self-Selection: Responders differ systematically from non-responders regarding both exposure prevalence and disease risk.
Design-Phase Prevention Strategies
Selection bias cannot be corrected during data analysis; it must be controlled during study design:
Formulate objective, explicit, and standardized definitions for cases, controls, exposed, and unexposed groups.
Enroll all eligible participants within a defined time frame and geographical setting.
Implement standardized procedures for recruiting all study groups.
Strive for high participation rates through non-coercive retention strategies and incentives.
Minimize loss to follow-up through active tracking protocols.
5. Information Bias, Misclassification, and Worked Mathematical Analysis
Information bias arises from systematic measurement flaws during data collection, leading to errors in classifying exposure or outcome status.
Forms of Information Bias
Information bias compromises data validity through several operational mechanisms:
Recall Bias: Cases recall past exposures differently (more or less accurately) than controls, a frequent challenge in retrospective case-control studies.
Interviewer Bias: Data collectors probe exposed or diseased subjects differently than unexposed or non-diseased subjects.
Reporting Bias: Participants selectively disclose or withhold sensitive exposure or medical information.
Surveillance/Monitoring Bias: Exposed participants receive closer medical monitoring, increasing the probability of disease detection relative to unexposed subjects.
Types of Misclassification
Misclassification represents the primary structural consequence of information bias:
Non-Differential Misclassification: Exposure misclassification is unrelated to disease status, or disease misclassification is unrelated to exposure status. The percentage error is equal across comparison groups, typically biasing effect estimates toward the null.
Differential Misclassification: Exposure misclassification depends on disease status, or disease misclassification depends on exposure status. The percentage error differs between groups, biasing effect estimates toward or away from the null.
Comprehensive Mathematical Worked Example
To demonstrate how misclassification alters risk estimates, consider a study evaluated using a standard 2 \times 2 contingency matrix:
\begin{array}{c|c|c} & \text{Disease (+)} & \text{No Disease (-)} \\ \hline \text{Exposed (+)} & a & b \\ \text{Unexposed (-)} & c & d \\ \end{array}
The Odds Ratio (OR) is calculated as:
OR = \frac{a \times d}{b \times c}
Step 1: True Baseline Data (Zero Misclassification)
The baseline true distribution of exposure and disease across 200 total subjects yields the following matrix:
Exposure Status | Disease (+) | No Disease (-) | Total |
Exposed (+) | a = 30 | b = 10 | 40 |
Unexposed (-) | c = 70 | d = 90 | 160 |
Total | 100 | 100 | 200 |
Calculating the true Odds Ratio (OR_{\text{true}}):
OR_{\text{true}} = \frac{30 \times 90}{70 \times 10} = \frac{2700}{700} \approx 3.86 \quad (\text{rounded in lecture to } 3.8)
Step 2: Non-Differential Misclassification Scenario
Suppose 30\% of all exposed participants are erroneously misclassified as unexposed, regardless of disease status (an equal 30\% error rate in both cases and controls):
Diseased Group (Cases): 30\% of 30 exposed cases (9 subjects) are misclassified as unexposed.
New Exposed Cases (a'): 30 - 9 = 21
New Unexposed Cases (c'): 70 + 9 = 79
Non-Diseased Group (Controls): 30\% of 10 exposed controls (3 subjects) are misclassified as unexposed.
New Exposed Controls (b'): 10 - 3 = 7
New Unexposed Controls (d'): 90 + 3 = 93
The resulting non-differential contingency matrix is:
Exposure Status | Disease (+) | No Disease (-) |
Exposed (+) | 21 | 7 |
Unexposed (-) | 79 | 93 |
Calculating the non-differential Odds Ratio (OR_{\text{nondiff}}):
OR_{\text{nondiff}} = \frac{21 \times 93}{79 \times 7} = \frac{1953}{553} \approx 3.53 \quad (\text{rounded in lecture to } 3.5)
Analytical Result: Because the error rate was equal across outcome groups (30\%), the calculated OR shifted from 3.8 to 3.5—biased toward the null.
Step 3: Differential Misclassification Scenario
Suppose 30\% exposure misclassification occurs only within the control group (0\% error in cases, 30\% error in controls):
Diseased Group (Cases): 0\% error. a'' = 30, c'' = 70.
Non-Diseased Group (Controls): 30\% of 10 exposed controls (3 subjects) misclassified as unexposed. b'' = 10 - 3 = 7, d'' = 90 + 3 = 93.
The resulting differential contingency matrix is:
Exposure Status | Disease (+) | No Disease (-) |
Exposed (+) | 30 | 7 |
Unexposed (-) | 70 | 93 |
Calculating the differential Odds Ratio (OR_{\text{diff}}):
OR_{\text{diff}} = \frac{30 \times 93}{70 \times 7} = \frac{2790}{490} \approx 5.69 \quad (\text{rounded in lecture to } 5.7)
Analytical Result: Because error rates differed between outcome groups (0\% vs. 30\%), the calculated OR shifted from 3.8 to 5.7—biased away from the null.
Strategies to Reduce Information Bias
Information bias must be addressed during study design and implementation:
Utilize objective, validated data collection instruments and clinical measures.
Develop a comprehensive Manual of Procedures (MOP) with standardized operational definitions.
Implement rigorous, standardized training and periodic re-training for data collection personnel.
Blind study participants and data collectors to exposure and disease hypotheses.
Employ structured memory aids and standardized questionnaire prompts to minimize recall distortion.
6. Methodological Synthesis: Phase of Control, Validity, and Precision
Achieving valid epidemiologic results requires aligning study design procedures with the specific timing constraints of error control mechanisms.
Phase of Control Matrix
Different types of study errors vary in when they can be controlled during the research timeline:
Category of Error | Study Design Phase | Data Analysis Phase |
Selection Bias | Controllable (Primary phase for control) | Cannot be corrected |
Information Bias | Controllable (Primary phase for control) | Cannot be corrected |
Random Sampling Error | Controllable (via sample size expansion) | Cannot be corrected |
Random Measurement Error | Controllable (via instrument protocol) | Cannot be corrected |
Confounding | Controllable (via restriction, matching, etc.) | Controllable (via stratification, regression) |
Key Methodological Takeaway: Selection and information biases cannot be repaired analytically once data collection completes. Uncontrolled biases must be reported as inherent study limitations. In contrast, confounding represents the sole systematic error that can be addressed during data analysis, provided appropriate confounder data were measured during the study.
Internal vs. External Validity and Precision
Study evaluation rests on three interconnected methodological criteria:
Internal Validity (Truth within the Study): Reflects the degree to which study results are free from methodological errors, selection bias, information bias, and uncontrolled confounding. Internal validity answers whether the observed effect estimate is accurate for the specific study sample. High internal validity is an indispensable baseline requirement for any research.
External Validity (Generalizability to Target Population): Reflects whether study conclusions can be validly applied to the broader target or general population. External validity depends on representative sampling from the source frame.
Precision (Reliability across Measurements): Refers to the reproducibility of study measurements, governed by the extent of random error and sample size.
Methodological Trade-offs in Study Design: Specialized research designs often trade external validity to maximize internal validity. For instance, randomized clinical trials enforce strict eligibility criteria to maximize internal validity (minimizing confounding and measurement error), but this narrow eligibility restricts sample representativeness, reducing external generalizability to broader patient populations.