Lecture 9: Experimental Design and Statistical Modeling: Fixed and Random Effects
Review of Statistical Progress and Module Structure
Course Trajectory:
Foundational Work: Early weeks involved basic statistical groundwork.
Linear Statistical Models (One Continuous Predictor): Taught by Anthony (previously Daniel). This involved one continuous response variable and one continuous predictor.
Week 1 (Module Extension): Multiple linear regression, extending to multiple continuous predictor variables.
Week 2: Transitioned from continuous to categorical predictors using Analysis of Variance (ANOVA).
Week 3: Combined continuous and categorical predictors (ANCOVA structures).
Variations on a Theme: The lecturer emphasizes that the fundamental linear model remains the same (often described as "plug and play"). The variations involve changes in assumptions, interpretations (e.g., bar charts for categorical vs. lines/planes for continuous), and experimental design.
Experimental Design Fundamentals: Experimental design and data analysis are interlinked. Key concepts covered include:
Replication and Randomization: Methods used to limit the influence of lurking variables.
Blocking Factors and Covariates: Used to explain specific portions of variation (reduce noise) to better identify the signal (the effect of interest).
Hypothesis Testing: Using ANOVA tables conceptually to test effects.
Introduction to Fixed and Random Effects
The Onion Metaphor: Learning statistics is described as "peeling back the layers of an onion." It becomes deeper and more complex as one progresses, and even experts are often "uncomfortable" or stumped by specific questions.
Fixed Effects (Example: Drug Trials):
Imagine a study on the effectiveness of three specific drugs: , , and .
Experimental Unit/Measurement Unit: 8 randomly selected patients are assigned to each drug. In this case, the patient is both the experimental unit (injected directly) and the measurement unit (e.g., measuring antibodies in the blood).
Independence: Because each patient is injected with one drug, the applications are independent.
Estimation and Hypothesis: In a fixed effect model, we estimate the mean effectiveness for each drug level (, , ). The hypothesis focuses on differences between these means.
Repetition: If the experiment is repeated with the same drugs but different patients, the resulting pattern (e.g., the "mustard-colored" drug being most effective) is expected to stay the same. Fixed effects are deliberate and fixed across repetitions.
Random Effects (Example: Clinics):
Introducing a structural feature: Clinics. Patients do not walk in from the street into one central location; they go to different clinics (e.g., Clinic , Clinic , etc.).
Non-independence (Clustering): Patients within the same clinic are not independent because they share a common environment (e.g., the same nurse, regional socioeconomic factors, or local exposure to illnesses like the flu). They are likely to respond similarly.
Estimation and Hypothesis: In a random effect model, we do not estimate a mean for each level (since clinics are randomly selected from a vast population). Instead, we estimate a variance (the spread between groups).
The Null Hypothesis: The null hypothesis for a random effect is that there is zero variance ().
Repetition: If the experiment is repeated, a different random set of clinics (, , ) would be selected from the population of thousands of possible clinics. The specific results would change because the clinics themselves are a random sample.
Nested vs. Blocked Designs
Nesting:
Definition: The levels of one factor occur within only a single level of another factor.
Example: Clinic nested within Drug. Clinic only receives Drug . Clinic only receives Drug . There is no crossover where Clinic also tests Drug .
Hierarchy: Patients are nested within Clinics, which are nested within Drugs.
Common Nested Examples:
Mice within cages, cells within mice.
Streams within catchments, reaches within streams.
Doctors within hospitals, hospitals within states.
Blocking:
Definition: Every level of the fixed factor (e.g., Drug) occurs within every level of the random factor (e.g., Clinic/Block).
Example: In a block design, Clinic would have some patients receiving Drug and other patients receiving Drug .
Distinguishing Method: Draw a matrix of the combinations. In a block design, most boxes for combinations of Drug and Clinic will be checked. In a nested design, only one drug level will appear per clinic.
Hierarchical (Multi-level) Models and Statistics
The Expanded Noise Term: In these models, random variation enters at different layers (hierarchies). The "noise" is no longer just . It contains:
Variation due to the random factor (e.g., Clinic).
Variation due to the residual error (e.g., Patient differences).
Word Equations (Conceptual):
Simple Design: .
Nested Design: .
Calculating the F-ratio:
In standard models, .
In nested designs with random factors, using the Residual Mean Square as the denominator is inappropriate because observations are not independent.
The true level of independent replication for the fixed effect is the higher-level unit (the Clinic).
Therefore: .
Specific Criteria for Fixed vs. Random Effects
Criterion | Fixed Effects | Random Effects |
|---|---|---|
Repetition | Same levels/groups used every time | Different groups selected from the same population |
Deliberation | Levels are chosen deliberately (e.g., specific COVID vaccines) | Levels are random (e.g., whichever clinics were available) |
Number of Levels | Generally a few known levels | Often many/unlimited levels to choose from |
Parameter Estimated | Estimating the Mean () | Estimating a Variance () or Standard Deviation |
Inference/Goal | Interested in specific differences between groups | Interested in variation introduced by the design |
Ambiguity: A factor can be either fixed or random depending on the study's goals. For example, Genotype is a fixed effect if the researcher is comparing specific genotypes, but it is a random effect if the researcher is randomly picking genotypes to see how plants generally respond to a heatwave.
Questions & Discussion
Question on Research Prerequisites: Is the current course a prerequisite for 3rd-year research projects?
Response: Yes, the lecturer believes it is one of the prerequisites. Research provides a "taste" of work, but the lecturer notes a labs capacity for projects may be limited compared to full-time Honours or PhD students who dedicate 1 to 3 years of work.
Question on F-ratio Calculation: Does the denominator of the F-ratio only use the specific nested level or a combined level?
Response: The F-ratio uses the Mean Square of that specific nested level () as the noise term for the level above it. It is hierarchical: "top over nested, nested over residuals."
Question on Definitions (ANOVA vs. Models):
Response: Confusion arises because "ANOVA" is often used to describe models with categorical predictors. Historically, ANOVA was taught as a distinct procedure, but it is simply a way of breaking up variation in a linear model.
Question on Nested Models vs. Nested Designs:
Response: In statistical modeling, a "nested model" describes a model that is a subset of another (e.g., Model 1 contains all terms of Model 2 plus an extra interaction term). This differs from a nested experimental design, which refers to the structural arrangement of the experiment (e.g., patients within clinics). Today's lecture focused on nested designs.