Selection Bias

Validity

  • Definitions of Validity

    • Accurate not Reliable

    • Valid

    • Reliable not Accurate

    • Precise

    • Accurate and Reliable

    • Valid and Precise

Types of Validity

External Validity

  • Defined as the generalizability or transportability of study results to other populations.

  • Critical considerations:

    • Question: To whom do our study results apply?

    • Question: Can we generalize beyond our study population?

    • Question: What caveats are there in our generalizations?

    • Importance of justification and argumentation regarding generalizability.

Internal Validity

  • Defined as the ability to produce unbiased estimates in the study.

  • Important considerations:

    • Question: Are our findings free of systematic error or bias?

    • Question: Was our study conducted rigorously to yield the correct result?

  • Factors affecting Internal Validity:

    • Selection Bias

    • Information Bias

    • Confounding

Understanding Bias

  • Definition of Bias

    • Any systematic error in an epidemiologic study that results in an incorrect estimate of the association between exposure and disease.

    • Reference: Hennekens and Buring, Epidemiology in Medicine

    • Reference: Kleinbaum, Kupper, and Morgenstern, Epidemiologic Research

    • Described as a distortion that may result when estimating the association of interest.

Types of Associations

  • Negative Associations: An occurrence where bias leads to an underestimation of risk.

  • Positive Associations: An occurrence where bias leads to an overestimation of risk.

Measure of Association

  • Ratio-based Measures:

    • Null Value: 1.0 (indicates no association)

    • Examples:

    • Null = 1.0

      • Observed Measurement: Reflects an association due to bias.

    • Truth = 2.0 can reflect a positive bias.

Direction of Bias

  • Bias Toward the Null:

    • Negative Associations: Null = 1.0, Observed = 1.5, Truth = 2.0

  • Bias Away from the Null:

    • Positive Associations: Null = 1.0, Truth = 1.5, Observed = 2.0

Selection Bias

  • Definition: Occurs when individuals selected or retained in a study distort the estimates of the truth.

  • Internal Validity Considerations:

    • Are the findings free of systematic error or bias?

    • The selection process might depend on:

    • Participant selection criteria

    • Subject-level factors influencing participation

    • Factors affecting continued participation in the study

    • Decisions made during data analysis stage

Hierarchy of Populations

  • Analyzed Population: Specific study question.

  • Target Population: External population to which results may be generalized.

  • Study Population: Individuals enrolled in the study.

  • Eligible Population: Intended sample for the study.

  • Source Population: Base population from which the study population is drawn.

Types of Selection Bias

  1. Selection of Participants: Participants selected into the study are not representative of the base population.

  2. Self-selection Bias: Individuals more likely to have the outcome self-select their exposure status.

  3. Selective Loss to Follow-Up: Participants who dropout from the study may share relationships with the exposure and outcome status.

    • Further subdivisions:

      • Attrition (right truncation)

      • Selective Response (left truncation)

Truth and Selection Bias

  • Examined based on the direction of inquiry regarding exposure, disease status, and the investigator's role in sample selection.

SWAN Study Selection

Mechanism 1 - Cross-Sectional Screening to Eligibility

  • Criteria for selection:

    • Age of 42-52 years, not on hormone therapy, not currently pregnant, intact uterus, at least 1 ovary, pre to peri-menopausal.

    • Cross-Sectional Screening:

    • Total Participants: n = 15,695

Mechanism 2 - Eligibility to Cohort Participation

  • Among eligible persons, selection based on interest to participate.

    • Longitudinal Cohort Finalized: n = 3,302

Figures and Tables

  • Table illustrating eligibility and participation proportions by racial/ethnic group and reasons for ineligibility in the SWAN study.

Self-Selection Bias

  • Definition: Arises when participants decide whether to join a study, creating differences between study subjects and the target population.

  • Factors influencing self-selection bias:

    • Voluntary participation

    • Level of health awareness

    • Access and motivation to participate

  • Case comparisons: Cases vs. controls, cohort volunteers vs. general population, and screening effects.

Examples of Self-Selection Bias

  • Healthy Worker Effect: Healthier individuals tend to be employed or volunteer, causing a disparity in morbidity rates against the general population.

  • Impact of Healthy Worker Effect:

    • If comparing a high exposure group (e.g., workers) versus the general population, the worker's baseline health will lead to an under-estimation of disease risk.

Example Scenario

  • If assessing lung cancer incidence among carpenters (high exposure to asbestos), their overall better health may make it appear that exposure is protective against lung cancer.

Selective Loss to Follow-Up

  • If participants lost to follow-up are more likely to be exposed, this biases the association in study data.

Affect on 2x2 Tables

  • The implications of outcome and non-outcome associations in those exposed versus not exposed.

Considerations in Randomized Controlled Trials (RCTs)

  • Factors contributing to selective loss, including treatment-related non-compliance and differential outcomes distribution.

Questions to Consider Regarding Selection Bias

  • Assess whether exposure is linked with the probability of inclusion for analysis.

  • Evaluate exposure's influence on the probability of follow-up outcomes.

  • Review the distribution of outcomes between included and excluded participants.

Recommendations to Mitigate Bias

  • Avoid adjusting for post-exposure variables.

  • Consider sequence timing of events in causal chains.

  • Utilize directed acyclic graphs (DAGs) as conceptual tools for clarity.

Final Advice

  • Strive to avoid biases at the design level whenever possible.

  • If it cannot be avoided, focus on targeted strategies to minimize impacts on study outcomes, such as restricting population inclusion and examining missing information on exposures.

  • Employ intent-to-treat analysis where applicable to maintain robust study conclusions.