causal interference

Causal Inference

Part 1: Understanding Cause and Effect

From Correlation to Causation

☑

Causal Inference Questions

• Will changing store environment help improve store traffic and sales?

• Will a revised promotional scheme work?

Causal Questions

• Primary focus is to determine if X causes Y

• Examples of variables:
    - X: Changes in the store environment, e.g., 10% off instead of $5 off
    - Y: Sales outcomes

Example 1: Causal Effect of Promotions

• Question: How do Christmas promotions affect sales?

Example 1: Promotion Data

• 
  

• Weekly sales data with and without Christmas promotions:
  

  Week	Sales ($ million)	X-mas Promo
  42	9.97	No
  43	8.17	No
  44	11.02	No
  45	9.97	No
  46	12.18	No
  47	8.17	No
  48	13.46	Yes
  49	16.44	Yes
  50	18.17	Yes
  51	16.44	Yes
  52	14.88	Yes
  Key Question: How can we estimate the effect of Christmas promotions on sales?
  Example 2: In-Store Music Ads & Sales

  • Question: Does playing audio ads in supermarkets increase sales across categories?

  Example 2: Music Ads and Store Sales

  • Method: Retail chain plays music-ads in some stores and compares sales to those without ads.

  • No Music Ad Stores: Average sales volume of 200 metric tons/store

  • With Music Ad Stores: Average sales volume of 400 metric tons/store

  • Implications: What is the effect of the music ad?

  Example 3: Bed Lamp Usage and Eye-Sight

  • Question: Does use of bed lamps cause eyesight to deteriorate?

  Example 3: Night Lamps & Eye Health

  • Reported Study (1999): Kids with lights on at bedtime are more likely to be short-sighted.

  • Inference: What conclusions can be drawn?

  Three Conditions for Causality

  Conditions for Establishing Causality

  1. Concomitant Variation
        - Variation in X (the choice) leads to variation in Y (the outcome). As X changes, Y should also change.

  2. Time Order of Occurrence
        - Change in X must occur before or simultaneously with the change in Y. The cause precedes the effect.

  3. Absence of Other Causal Factors
        - No confounding variables should affect the relationship between X and Y, meaning the relationship is not driven by a third variable Z.

  Confounding Problems

  Example 1: Christmas Promo & Sales

  • Issue: The festive period drives both promotions AND higher sales, thus acting as a confounder.

  Example 2: Music Ads

  • If the chain chooses to play music ads in higher-selling stores, a systematic difference emerges:
      - No Ad Stores might be smaller or located in lower-income areas with already low foot traffic.
      - With Music Ad Stores may be larger, in higher-income areas with high foot traffic.
      - Result: The 100% difference in average sales reflects store characteristics instead of the impact of music ads.

  Example 3: Night Lights & Short-Sightedness

  • Initial Study (1999): It was concluded that night lights caused short-sightedness.

  • Follow-up Study (2000): Genetic conditions acted as a confounding variable; short-sighted kids may use more night lights — this indicates potential reverse causality.

  Correlation vs. Causation

  Key Concept: Counterfactual Thinking

  • The core question: What would have happened if the groups were in the alternative condition?

  The Counterfactual Problem

  • Need for counterfactual estimates exists, despite being limited to observing one outcome per group.
      - Without Music Ads (A): ??? Metric Tons
      - With Music Ads (B): 400 Metric Tons
      - We can never observe both potential outcomes for the same group; this represents the fundamental issue within causal inference.

  Estimating Counterfactual Outcomes

  • If missing outcomes could be estimated, the true causal effect could be calculated.
      - Example of estimates:
      - Without Music Ads (A): 200 Metric Tons -> 250 Metric Tons (estimation)
      - With Ads (B): 300 Metric Tons -> 400 Metric Tons
      - Observed effect would be +50 Metric Tons from no ads and +100 Metric Tons from treatment, but how to generate these counterfactual estimates?

  The Solution: A/B Tests

  A/B Tests Explained

  • Also referred to as Randomized Controlled Trials.

  • Framework: Randomly assign similar entities into two groups, allowing the isolation of the causal effect.

  • Control Group: Receives no intervention (No music ads played).

  • Treatment Group: Receives the intervention (Music ads played).

  • Randomization ensures both groups are similar, ensuring that differences in outcomes are attributable to the treatment.

  Why Randomization Works

  • Random assignment of music ads leads to comparable groups:
      - Control Group: No Music
      - Treatment Group: Music Ads

  • Each group's average can be used as the counterfactual for the other due to their similarity from random assignment.

  Randomized Results: True Effect

  • Without Ads (A): 300 Metric Tons

  • With Ads (B): ??? Metric Tons (estimate based on group similarity)

  • Effect Calculation: Using averages, estimate the counterfactual for both groups.

  Framework: A/B Test Steps

  Steps in Conducting A/B Tests

  1. Define the Intervention: Specify the treatment to apply (e.g., music ads played in store).

  2. Identify Test Units: Select stores or entities for the experiment.

  3. Randomly Assign to Control & Treatment Groups: Assign stores randomly to avoid bias.

  4. Check Similarity Between Groups: Ensure groups are similar in relevant characteristics.

  5. Estimate Causal Effect: Compare outcomes (e.g., sales) across groups post-intervention.

  Applied A/B Test Steps to Music Ad Example

  1. Define Intervention: Implementation of music ads in stores.

  2. Identify Test Units: Selection of stores for the experiment.

  3. Random Assignment: Choose which stores receive ads and which do not.

  4. Check Similarity: Compare characteristics like store size and neighborhood demographics between groups.

  5. Estimate Effect: After the interventions, compare sales figures between treatment and control groups.

  Checking Randomization

  Randomization Balance Assessment

  • Compare average characteristics (not outcomes) of the treatment and control groups using two primary methods:

  1. T-Test
        - Assess averages of both groups.
        - p-value > 0.05 indicates groups are similar, affirming randomization holds.
        - p-value ≤ 0.05 suggests the groups differ, requiring a re-evaluation of random assignment.

  2. Standardized Mean Difference (SMD)
        - Expresses the difference in means in pooled standard deviation units:
        - SMD < 0.1 signifies good balance between groups.

  SMD Formula

  • To calculate SMD for each characteristic, use:
    $\text{SMD} = \frac{\text{Average value for treatment group} - \text{Average value for control group}}{\text{pooled std. dev.}}$

  • Where Pooled Standard Deviation is given by:
    $s_p = \sqrt{\frac{\text{treatment group variance} + \text{control group variance}}{2}}$

  Estimating the Causal Effect

  • Compare the outcome of interest (e.g., sales) between the two groups post-intervention:

  • p-value > 0.05:
       - Indicates no statistically significant effect of the treatment, concluding music ads had no impact on sales.

  • p-value ≤ 0.05:
       - Provides evidence of a significant treatment effect, indicating music ads positively influenced sales.

  In-Class Exercise

  Hands-on Analysis

  • Open: experiment_(post test only).xlsx

  • Apply the A/B test framework to analyze experimental data.

  • Tasks: Check randomization, estimate the causal effect based on comparisons.

  Key Takeaways

  • Correlation does not imply causation; confounders can create spurious relationships.

  • Causality requires:
      - Concomitant Variation
      - Time Order
      - Absence of Confounders

  • Counterfactual thinking serves as a foundation for causal inference.

  • Randomized Controlled Trials (A/B tests) represent the gold standard for causal inference.

  • Always check balance in randomization using t-tests or SMD before estimating effects.