AP Statistics Notes: Experimental Design, Blocking, and Statistical Inference

Variables, Confounding, and Experimental Setup

  • Response Variable:

    • Measures the outcome of a study or experiment.

  • Explanatory Variable:

    • May explain or predict changes observed in the response variable.

  • Factor:

    • An explanatory variable that is actively manipulated by the experimenter.

  • Levels:

    • The different specific values or settings used for a given factor.

  • Confounding:

    • Occurs when two variables are associated in such a way that their individual effects on a response variable cannot be distinguished from one another.

  • Three-Step Framework for Addressing Confounding:

    • Step 1: Identify the confounding variable and explain explicitly how it was not controlled in the experiment.

    • Step 2: Explain the association between the confounding variable and both the explanatory variable and the response variable.

    • Step 3: State clearly what direction this confounding variable will skew the experimental results.

  • ATFQ Strategy:

    • "Answer The Fu----- Question": Explicitly address every component asked in the prompt without leaving steps out.

  • Case Analysis (Sacramento Kings Crowd Size and Performance):

    • Questioning cause and effect: It is not reasonable to conclude that smaller crowds directly cause better team performance.

    • Confounding explanation: It is plausible that the Kings were playing a worse team, resulting in lower fan attendance. Playing a weaker opponent allowed an environment for the Kings to win more games compared to when they played better teams. Thus, their better winning record occurred simultaneously with smaller crowds due to opponent quality.

  • Treatment:

    • A specific condition applied to experimental units in an experiment.

    • Formed by a specific combination of factors and levels.

  • Experimental Unit:

    • The object or individual to which a treatment is applied.

    • When human, experimental units are referred to as "subjects" or "participants".

  • Adhesive Batch Scenario:

    • Experimental units: Batches of adhesive.

    • Number of experimental units: 1212.

    • Response variable: Kilos of useable cohesive.

    • Factors: Temperature (Temp\text{Temp}) and stirring speed.

    • Number of treatments: 66

  • Placebo and Placebo Effect:

    • Placebo: A "dummy" treatment that is physically indistinguishable from active treatments.

    • Placebo effect: The occurrence where subjects respond to any treatment, even an inactive dummy treatment.

  • Blinding Protocols:

    • Single-blind: Either the subject or the experimenter knows which treatment was assigned, but not both.

    • Double-blind: Neither the subject nor the experimenter collecting observations knows which treatment was received.

Principles of Experimental Design

  • Control Group:

    • A group used to provide a baseline for comparing the effects of other treatments.

  • Random Assignment:

    • Treatments are assigned to experimental units using a chance process.

    • Ensures that treatment groups are roughly equal in all respects prior to treatment application (except for the treatment itself).

  • Replication:

    • Use enough experimental units in each group so that variability between treatment groups can be reduced and natural variability between groups can be estimated.

  • Control:

    • Keep all variables other than the treatment variable the same across all groups.

  • Four Core Principles of Experimental Design:

    1. Comparison

    2. Random assignment

    3. Replication

    4. Control

Experimental Design Methodologies

  • Completely Randomized Design:

    • Definition: An experimental design in which treatments are assigned to experimental units (or vice-versa) entirely using a chance process.

    • Standard Protocol:

      • Label: Write experimental units on cards (number units 11 to NN, number cards 11 to NN).

      • Randomize: Shuffle cards thoroughly.

      • Select: Deal out cards to assign units to treatments.

    • Detailed Example (12001200 Participants):

      • Label: Prepare 300300 cards with "injection", 300300 cards with "pill", 300300 cards with "injection + pill", and 300300 cards with "nothing".

      • Randomize: Place all 12001200 cards into a hat and mix well.

      • Select: Deal out the 12001200 cards to the 12001200 participants.

  • Randomized Block Design:

    • Definition: Experimental units are grouped into blocks of units that are similar in some key way that might affect results. Random assignment of treatments is carried out independently within each block.

    • Standard Protocol:

      • Label: Within each block, label units and cards 11 to N$.\n * Randomize: Shuffle each stack of cards separately.\n * Select: Within each block, deal out cards for treatment 1,treatment, treatment2, etc.\n * Detailed Example (Apple Varieties and Fertilizer):\n * Label: Group by variety and label units: Honeycrisp (1\text{--}40),Sugarbee(), Sugarbee (1\text{--}30),RedDelicious(), Red Delicious (1\text{--}20).\n * Randomize: Label each stack of cards and shuffle them separately.\n * Select: Select 20HoneycrispfornewfertilizerandHoneycrisp for new fertilizer and20foroldfertilizer;selectfor old fertilizer; select15SugarbeefornewfertilizerandSugarbee for new fertilizer and15foroldfertilizer;selectfor old fertilizer; select10RedDeliciousfornewfertilizerandRed Delicious for new fertilizer and10 for old fertilizer.\n\n* **Matched Pairs Design**:\n * Definition: A specialized randomized block design with a block size of 2$.

    • Structural Options:

      • 22 similar individuals (e.g., twins, or 22 sides of the same individual).

      • Use the same individual receiving both treatments in a randomized order.

    • Standard Protocol:

      • Label: Label each pair and write corresponding labels on cards.

      • Randomize: Shuffle cards.

      • Select: Deal to determine which individual in a pair receives treatment 11 or treatment 2$.\n * Detailed Example (Employee Interface/Face Study):\n * Label: Label each employee and cards 1\text{--}40$.

      • Randomize: Shuffle cards.

      • Select: Deal out 2020 cards. Those 2020 employees get new faces on the right side and old faces on the left side. The other 2020 employees get the reverse configuration (flip overs).

Strategic Controlling of Variables

  • Advantages of Blocking and Matched Pairs:

    • Primary Goal: Reduces variability, making it easier to infer cause and effect.

    • Contextual Connection: Always explain in context how specific blocks or pairs affect results.

    • Proportional Representation: Ensures the exact same proportion of individuals with a given trait exists within each treatment group.

    • Cause and Effect Link: Reduced variability makes it easier to gather strong evidence and establish true cause and effect.

  • Three-Tier Variable Control Strategy:

    1. Control what you can.

    2. Block what you cannot control.

    3. Randomize the rest.

Statistical Notation, Hypotheses, and Inference Types

  • Difference of Means vs. Mean Difference:

    • Difference of Means (Independent Samples):

      • Population Parameter: μ1−μ2\mu_1 - \mu_2

      • Sample Statistic: xˉ1−xˉ2\bar{x}_1 - \bar{x}_2

      • Standard Deviation Notation: s1−s2s_1 - s_2 or σ1−σ2\sigma_1 - \sigma_2

    • Mean Difference (Paired Data / Matched Pairs):

      • Population Parameter: μdiff\mu_{\text{diff}}

      • Sample Statistic: xˉdiff\bar{x}_{\text{diff}}

      • Standard Deviation Notation: sdiffs_{\text{diff}}

  • Hypothesis Formulation:

    • Null Hypothesis (H0H_0, "H-naught"): Random assignment accounts for the observed difference between treatment and control groups (treatment had no true effect).

    • Alternative Hypothesis (HaH_a): The treatment actually had an effect.

  • Statistical Significance:

    • When results are statistically significant, we reject the null hypothesis (H0H_0) and conclude that there is convincing evidence for the alternative hypothesis (HaH_a).

  • Types of Inference Rules:

    • Random Selection: Allows inferences about the broader population.

    • Random Assignment: Allows inference about cause and effect.