Statistical Study Designs: Experiments, Observational Studies, and Confounding Variables

Fundamentals of Experimental Design

• Definition and Purpose of Statistical Experiments:

  • Statistical experiments are structured research studies carried out to systematically answer specific research questions.

  • Example Research Question: Administering $35\,\text{mg}$ of aspirin daily to determine whether it reduces the overall risk of heart attacks.

  • Experiments allow researchers to manipulate independent variables and evaluate direct causal effects.

• Four Primary Characteristics of Experimental Design:

  • 1. Control:

  • Experiments require a clear comparative structure, consisting of a treatment group and a control group.

  • The treatment group receives the actual active treatment or intervention being evaluated.

  • The control group receives a placebo, which is a baseline, inactive treatment.

  • Purpose of Placebos: Giving a placebo to the control group evens out potential psychological or mental placebo effects across all subjects. If control subjects know they are not receiving an active medication, psychological factors can negatively bias the outcome. Placebos ensure all subjects share identical expectations while researchers track group allocations.

  • Prevention of Bias: Subjects must be randomly assigned to prevent researcher bias, such as intentionally placing family members or healthier subjects into a specific group.

  • 2. Randomization:

  • Subjects are randomly selected from the general population and subsequently randomly assigned to experimental treatment groups.

  • Mechanisms for Random Assignment:

    • Die Rolling: Rolling a fair six-sided die where landing on an even face assigns the subject to the treatment group, and landing on an odd face assigns the subject to the control group. Because the outcome cannot be predicted in advance, fairness is guaranteed.

    • Coin Tossing: Tossing a fair coin where landing on Heads assigns the subject to the treatment group, and landing on Tails assigns the subject to the control group.

  • 3. Replication:

  • Replication involves repeating the experimental process over an extended period or across a large number of trials to ensure consistent, stable, and generalizable results.

  • Implementation via Sample Size: Directly repeating an entire multi-year study multiple times is often impractical; thus, replication is frequently operationalized by expanding the sample size ($n$) of participants.

  • Increasing an experimental sample size from $n = 50$ subjects to $n = 200$ subjects mathematically equates to performing the experimental procedure $200$ individual times, provided adequate resources exist.

  • 4. Blocking:

  • Blocking is the procedure of partitioning subjects into homogenous subsets (blocks) based on known, non-manipulated characteristics prior to treatment assignment.

  • Peer Influence Scenario: In a study on smoking behaviors, two close friends ("buddies") might influence each other's choices (e.g., one persuading the other not to smoke a particular cigarette brand). To eliminate peer interference, such individuals are separated into distinct blocks before being randomly assigned to treatment conditions.

Limitations of Experimental Studies

• Ethical Constraints in Research:

  • Not all research questions can be investigated using randomized controlled trials or double-blind experiments due to ethical boundaries.

  • Smoking and Lung Cancer Example:

  • Research Question: Does smoking cause lung cancer?

  • To study this question via a randomized experiment, researchers would have to randomly assign human subjects to a treatment group and force them to smoke high volumes of cigarettes over extended periods.

  • Forcing subjects to participate in potentially harmful behaviors is strictly prohibited on ethical grounds.

  • Consequently, studies involving hazardous habits rely on non-experimental designs, unless individuals independently volunteer to participate while already engaging in the behavior.

• Practical and Methodological Constraints:

  • Certain conditions make constructing a viable control group or realistic placebo impossible (e.g., simulating invasive surgical interventions or long-term structural lifestyle changes).

Principles of Observational Studies

• Definition of Observational Studies:

  • An observational study refers to a research design where data are gathered passively without direct interference or variable manipulation by researchers.

  • Researchers observe naturally occurring conditions and recorded behaviors without imposing artificial conditions.

  • Classroom Inspection Metaphor: An inspector evaluating student classroom behavior sits at the back of the room without alerting the students. Natural interactions occur (such as a student named Peter taking a phone call from his mother saying, "Hey mom, I'm in class," or another student reading a magazine). If students were explicitly told they were being monitored, their behavior would change. Unannounced passive observation preserves natural behavioral data.

• Key Distinctions: Experiments vs. Observational Studies:

  • Variable Interference: Experiments actively manipulate explanatory variables; observational studies record data naturally as they occur.

  • Causal Conclusions:

  • Experiments: Permitted to establish direct causal relationships ("cause-and-effect") due to controlled conditions and random assignment.

  • Observational Studies: Restricted strictly to identifying associations or correlations between variables. Direct causal conclusions can never be drawn from observational studies.

• Identifying Variables in Research Prompts:

  • Response Variable: The outcome variable measured to observe the result or effect (e.g., lung cancer status, visual acuity).

  • Explanatory Variable: The variable hypothesized to cause, predict, or explain changes in the response variable (e.g., smoking status, nighttime lighting exposure).

  • Locating Variables in Written Summaries: Explanatory and response variables are typically introduced in the initial two sentences of a research prompt or explicitly detailed in the final conclusion section.

Confounding Variables and Causal Limitations

• Definition of a Confounding Variable:

  • An unmeasured, underlying variable that exerts an influence on both the explanatory variable and the response variable, producing a false or misleading association between them.

• The Dual-Relationship Criterion for Confounding Variables:

  • To qualify as a valid confounding variable, a candidate variable ($C$) must hold a direct relationship with both:

  1. The explanatory variable ($A$).

  2. The response variable ($B$).

  • If a variable relates solely to the explanatory variable or solely to the response variable, it fails the criterion and is not a confounding variable.

• Case Study 1: Nighttime Lighting and Childhood Myopia:

  • Research Question: Are infants raised without exposure to total darkness more likely to suffer from myopia (short-sightedness) later in life?

  • Study Protocol: Researchers recorded ambient room lighting during infancy and re-evaluated the vision of the same children approximately $10$ years later.

  • Variables:

  • Explanatory Variable: Nighttime lighting exposure (darkness vs. active light) during infancy.

  • Response Variable: Myopia diagnosis in later childhood.

  • Identified Confounding Variable: Parental Vision / Parental Myopia.

  • Verification of Confounding Criterion:

  • Relationship to Explanatory Variable: Myopic parents are far more likely to keep nightlights on in their infants' rooms to see clearly when tending to them during the night.

  • Relationship to Response Variable: Myopia is a genetically inherited trait; children of myopic parents have an elevated genetic likelihood of developing short-sightedness.

  • Conclusion: Parental vision influences ambient lighting choices and childhood visual health. Because parental vision was unobserved in the initial model, researchers cannot conclude that nighttime lighting causes myopia.

• Case Study 2: Sunscreen Usage and Skin Cancer Rates:

  • Observational Finding: Increased frequency of sunscreen usage is statistically associated with an increased incidence of skin cancer.

  • Variables:

  • Explanatory Variable: Amount or frequency of sunscreen applied.

  • Response Variable: Diagnosis of skin cancer.

  • Identified Confounding Variable: Total time spent outdoors in direct sunlight.

  • Verification of Confounding Criterion:

  • Relationship to Explanatory Variable: Individuals spending extended hours in direct sunlight apply significantly more sunscreen.

  • Relationship to Response Variable: Prolonged solar ultraviolet (UV) exposure directly elevates the risk of developing skin cancer.

  • Conclusion: Time in the sun drives both sunscreen application and skin damage. It acts as a confounding variable, rendering any conclusion that sunscreen causes skin cancer invalid.

Questions and Discussion

• Testing Candidate Confounding Variables:

  • Discussion Point: How should one systematically verify if a candidate variable is a confounding variable?

  • Methodology: Apply the formal template: "The proposed confounding variable is [Variable $C$], and it exerts a demonstrable effect on both [Explanatory Variable $A$] and [Response Variable $B$]."

  • Example Application (Sleep Quality and Duration):

  • Research Context: Evaluating drivers of sleep length and sleep quality.

  • Proposed Confounding Variables: Exercise habits and work schedules.

  • Exercise Habits: Directly modifies sleep duration and independently affects health outcomes.

  • Work Schedule: Constrains available sleep hours and independently impacts stress levels and physical wellness.

  • Both exercise habits and work schedule satisfy the dual-relationship test and function as legitimate confounding variables.

• Utility of Observational Studies:

  • Discussion Point: If causal conclusions cannot be drawn, what is the core utility of observational studies?

  • Core Utility:

  • Observational studies provide valuable real-world evidence when experimental interventions are unfeasible or unethical (e.g., assessing health impacts of exposure to toxic substances or infant antibiotic treatments).

  • They establish initial relationships and highlight statistical associations ("smell it right" indicators) that guide future targeted research.

• Structural Summary Matrix:

  • Observational Studies:

  • Data collected passively without intervention.

  • Explanatory variables are not manipulated.

  • Random assignment cannot be applied.

  • Vulnerable to unmeasured confounding variables.

  • Establishes statistical association only; cannot prove causation.

  • Experimental Studies:

  • Explanatory variables are directly manipulated with applied treatments.

  • Random assignment is used to distribute confounding variables evenly.

  • Utilizes control groups, placebos, and blocking to control extraneous factors.

  • Establishes valid cause-and-effect conclusions.