Lecture 9: Experimental Design and Statistical Modeling: Fixed and Random Effects

Review of Statistical Progress and Module Structure

  • Course Trajectory:

    • Foundational Work: Early weeks involved basic statistical groundwork.

    • Linear Statistical Models (One Continuous Predictor): Taught by Anthony (previously Daniel). This involved one continuous response variable and one continuous predictor.

    • Week 1 (Module Extension): Multiple linear regression, extending to multiple continuous predictor variables.

    • Week 2: Transitioned from continuous to categorical predictors using Analysis of Variance (ANOVA).

    • Week 3: Combined continuous and categorical predictors (ANCOVA structures).

  • Variations on a Theme: The lecturer emphasizes that the fundamental linear model remains the same (often described as "plug and play"). The variations involve changes in assumptions, interpretations (e.g., bar charts for categorical vs. lines/planes for continuous), and experimental design.

  • Experimental Design Fundamentals: Experimental design and data analysis are interlinked. Key concepts covered include:

    • Replication and Randomization: Methods used to limit the influence of lurking variables.

    • Blocking Factors and Covariates: Used to explain specific portions of variation (reduce noise) to better identify the signal (the effect of interest).

    • Hypothesis Testing: Using ANOVA tables conceptually to test effects.

Introduction to Fixed and Random Effects

  • The Onion Metaphor: Learning statistics is described as "peeling back the layers of an onion." It becomes deeper and more complex as one progresses, and even experts are often "uncomfortable" or stumped by specific questions.

  • Fixed Effects (Example: Drug Trials):

    • Imagine a study on the effectiveness of three specific drugs: A1A_1, A2A_2, and A3A_3.

    • Experimental Unit/Measurement Unit: 8 randomly selected patients are assigned to each drug. In this case, the patient is both the experimental unit (injected directly) and the measurement unit (e.g., measuring antibodies in the blood).

    • Independence: Because each patient is injected with one drug, the applications are independent.

    • Estimation and Hypothesis: In a fixed effect model, we estimate the mean effectiveness for each drug level (Drug A1\text{Drug } A_1, Drug A2\text{Drug } A_2, Drug A3\text{Drug } A_3). The hypothesis focuses on differences between these means.

    • Repetition: If the experiment is repeated with the same drugs but different patients, the resulting pattern (e.g., the "mustard-colored" drug A3A_3 being most effective) is expected to stay the same. Fixed effects are deliberate and fixed across repetitions.

  • Random Effects (Example: Clinics):

    • Introducing a structural feature: Clinics. Patients do not walk in from the street into one central location; they go to different clinics (e.g., Clinic B1B_1, Clinic B2B_2, etc.).

    • Non-independence (Clustering): Patients within the same clinic are not independent because they share a common environment (e.g., the same nurse, regional socioeconomic factors, or local exposure to illnesses like the flu). They are likely to respond similarly.

    • Estimation and Hypothesis: In a random effect model, we do not estimate a mean for each level (since clinics are randomly selected from a vast population). Instead, we estimate a variance (the spread between groups).

    • The Null Hypothesis: The null hypothesis for a random effect is that there is zero variance (Variance=0\text{Variance} = 0).

    • Repetition: If the experiment is repeated, a different random set of clinics (B4B_4, B5B_5, B6B_6) would be selected from the population of thousands of possible clinics. The specific results would change because the clinics themselves are a random sample.

Nested vs. Blocked Designs

  • Nesting:

    • Definition: The levels of one factor occur within only a single level of another factor.

    • Example: Clinic nested within Drug. Clinic B1B_1 only receives Drug A1A_1. Clinic B3B_3 only receives Drug A2A_2. There is no crossover where Clinic B1B_1 also tests Drug A2A_2.

    • Hierarchy: Patients are nested within Clinics, which are nested within Drugs.

    • Common Nested Examples:

      • Mice within cages, cells within mice.

      • Streams within catchments, reaches within streams.

      • Doctors within hospitals, hospitals within states.

  • Blocking:

    • Definition: Every level of the fixed factor (e.g., Drug) occurs within every level of the random factor (e.g., Clinic/Block).

    • Example: In a block design, Clinic B1B_1 would have some patients receiving Drug A1A_1 and other patients receiving Drug A2A_2.

  • Distinguishing Method: Draw a matrix of the combinations. In a block design, most boxes for combinations of Drug and Clinic will be checked. In a nested design, only one drug level will appear per clinic.

Hierarchical (Multi-level) Models and Statistics

  • The Expanded Noise Term: In these models, random variation enters at different layers (hierarchies). The "noise" is no longer just Error\text{Error}. It contains:

    • Variation due to the random factor (e.g., Clinic).

    • Variation due to the residual error (e.g., Patient differences).

  • Word Equations (Conceptual):

    • Simple Design: Response=Drug+Unexplained Variation\text{Response} = \text{Drug} + \text{Unexplained Variation}.

    • Nested Design: Response=Drug+Clinic(Drug)+Error\text{Response} = \text{Drug} + \text{Clinic(Drug)} + \text{Error}.

  • Calculating the F-ratio:

    • In standard models, F=Mean Square (Signal)Mean Square (Residual){F} = \frac{\text{Mean Square (Signal)}}{\text{Mean Square (Residual)}}.

    • In nested designs with random factors, using the Residual Mean Square as the denominator is inappropriate because observations are not independent.

    • The true level of independent replication for the fixed effect is the higher-level unit (the Clinic).

    • Therefore: F=MSFixed EffectMSNested Random Factor{F} = \frac{MS_{\text{Fixed Effect}}}{MS_{\text{Nested Random Factor}}}.

Specific Criteria for Fixed vs. Random Effects

Criterion

Fixed Effects

Random Effects

Repetition

Same levels/groups used every time

Different groups selected from the same population

Deliberation

Levels are chosen deliberately (e.g., specific COVID vaccines)

Levels are random (e.g., whichever clinics were available)

Number of Levels

Generally a few known levels

Often many/unlimited levels to choose from

Parameter Estimated

Estimating the Mean (xˉ\bar{x})

Estimating a Variance (SD2\text{SD}^2) or Standard Deviation

Inference/Goal

Interested in specific differences between groups

Interested in variation introduced by the design

  • Ambiguity: A factor can be either fixed or random depending on the study's goals. For example, Genotype is a fixed effect if the researcher is comparing specific genotypes, but it is a random effect if the researcher is randomly picking genotypes to see how plants generally respond to a heatwave.

Questions & Discussion

  • Question on Research Prerequisites: Is the current course a prerequisite for 3rd-year research projects?

    • Response: Yes, the lecturer believes it is one of the prerequisites. Research provides a "taste" of work, but the lecturer notes a labs capacity for projects may be limited compared to full-time Honours or PhD students who dedicate 1 to 3 years of work.

  • Question on F-ratio Calculation: Does the denominator of the F-ratio only use the specific nested level or a combined level?

    • Response: The F-ratio uses the Mean Square of that specific nested level (MSClinicMS_{\text{Clinic}}) as the noise term for the level above it. It is hierarchical: "top over nested, nested over residuals."

  • Question on Definitions (ANOVA vs. Models):

    • Response: Confusion arises because "ANOVA" is often used to describe models with categorical predictors. Historically, ANOVA was taught as a distinct procedure, but it is simply a way of breaking up variation in a linear model.

  • Question on Nested Models vs. Nested Designs:

    • Response: In statistical modeling, a "nested model" describes a model that is a subset of another (e.g., Model 1 contains all terms of Model 2 plus an extra interaction term). This differs from a nested experimental design, which refers to the structural arrangement of the experiment (e.g., patients within clinics). Today's lecture focused on nested designs.