Study Notes on Causal Inference and Experimental Design

Causal inference is the study of determining cause-and-effect relationships from data, often moving beyond mere associations to establish why certain outcomes occur. It seeks to answer "what if" questions assuming an intervention.
A counterfactual statement is an if-then statement about an event that hasn't occurred. For example, "If I had taken the medicine (contrary to fact), then my headache would have disappeared," highlighting the core concept of comparing observed outcomes with unobserved potential outcomes under different conditions.

Causal questions involve hypothetical interventions and manipulation of variables (treatments) to explore their direct impact on outcomes. They typically ask: "What would happen if we did X?" or "Does X cause Y?"
Statistical questions, on the other hand, are about observing and describing existing correlations, patterns, and relationships in data without necessarily inferring cause. They might ask: "Is X associated with Y?" or "How strong is the relationship between X and Y?"

Correlation signifies that two variables change together, but it definitively does not imply causation. This is a fundamental principle in statistical analysis.
A strong statistical correlation between two variables, while visually compelling, does not guarantee that one variable directly causes the other. It is a necessary but not sufficient condition for establishing causality; other factors, like lurking variables or confounders, could be driving the observed relationship.

A confounder is an extraneous variable that influences both the independent variable (treatment, $x$ ) and the dependent variable (response, $y$ ), thereby creating a spurious association between $x$ and $y$ . Failing to account for confounders can lead to incorrect causal conclusions. For instance, in a study on coffee and heart disease, smoking could be a confounder.
A spurious correlation exists when two variables appear related or correlated due to coincidence or the presence of an unobserved third variable (a confounder), rather than a direct, causal link between them. These correlations can be statistically significant but causally meaningless, such as ice cream sales and shark attacks.

Control Group: An indispensable component of a well-designed experiment. The control group does not receive the treatment (or receives a placebo) and serves as a baseline for comparison against the treatment group. This helps isolate the effect of the treatment by ruling out natural variation, time effects, or the placebo effect.
Blocking: A technique used to reduce unwanted variation. It involves dividing experimental units into homogeneous subgroups (blocks) based on known characteristics that might affect the outcome. Treatments are then randomly assigned within each block, ensuring that comparisons are made among similar units, which increases the precision of the experiment.
Randomization: The cornerstone of causal inference in experiments. It involves randomly assigning participants or experimental units to either the treatment group or the control group. This process helps distribute both known and unknown confounding variables equally across groups, ensuring that any observed differences in outcomes are more likely attributable to the treatment rather than pre-existing differences between the groups.

The placebo effect refers to a phenomenon where a patient experiences a real physiological or psychological improvement after receiving an inert treatment (a placebo) simply because they believe they are receiving an active treatment. This highlights the powerful influence of expectation and belief on health outcomes and underscores why control groups receiving placebos are crucial in clinical trials to genuinely assess drug efficacy.

Random assignment of participants to different experimental groups is critical because it ensures that, on average, all potential confounding variables (both observed and unobserved) are equally distributed across these groups. This balance makes the groups comparable, allowing researchers to confidently infer that any post-treatment differences between groups are due to the treatment itself and not to other pre-existing factors.

While randomization is powerful, it does not eliminate confounding variables; rather, it balances their influence across treatment and control groups, effectively neutralizing their impact on the observed treatment effect. It creates statistically equivalent groups at the start of the experiment.
Systematic errors (bias) are consistent, repeatable errors that occur due to flaws in the experimental design, measurement instruments, or the observational process. They push measurements consistently in one direction (e.g., a miscalibrated scale always reading too high). Systematic errors affect accuracy.
Random errors are unpredictable, chance variations that arise from uncontrollable factors or inherent variability in natural processes. They cause measurements to fluctuate around the true value without a consistent pattern (e.g., slight variations in reading a dial). Random errors affect precision.

Blocking is a sophisticated experimental design technique that enhances the precision and statistical power of an experiment by accounting for variation attributable to specific, identifiable factors (e.g., age, gender, location). By grouping similar experimental units into blocks, we can isolate the treatment effects from the variability caused by these nuisance factors, thereby increasing the signal-to-noise ratio and allowing for more accurate and robust causal inferences.