Repeated Measures Designs: Advantages, Disadvantages, and Statistical Utility, and Analysis

Comparison of Research Designs and the Repeated Measures Approach

• Repeated measures designs are fundamentally contrasted with between-groups (independent groups) designs based on how participants interact with the independent variable (IV).
• In a repeated measures approach, the participant (or an object) is subjected to all levels or factors of the independent variable, and the dependent variable (DV) is measured for every level.
• In between-groups designs, each participant is subjected to only one level of the manipulation, resulting in a single measurement per participant.
• The independent variable in these contexts is often referred to as a factor, a term particularly useful when discussing factorial designs.
• Similar to between-groups designs, repeated measures can involve $2$ or more levels. These levels may include:     * Control conditions versus experimental conditions.     * Quantitative increases, such as an increased dosage of a substance.     * Categorical manipulations, representing completely different types of conditions.
• Terminology notes: While between-groups research refers to groups of participants, repeated measures research uses the generic term levels or the specific term conditions, as there are no distinct groups.
• Psychology is unique among scientific research areas for its heavy emphasis on and frequent use of repeated measures designs compared to other disciplines.

Categorization of Repeated Measures in Psychology

• Repeated measures designs in psychology generally fall into two categories:     * Experimental Research: Commonly found in cognitive psychology, where each level of the IV is experimentally manipulated. In these designs, the sequence of levels can be varied without affecting the validity of the results. Examples include varying stimulus intensity, different types of stimuli, or levels of stress (e.g., $0$ stress, mild stress, and high stress). The order of these sequences is not fixed and can differ between participants.     * Longitudinal Research: In this area, the sequence of the levels is fixed and matters significantly. Each level typically represents a quantitative, chronological change. Examples include measurements across Age ($2$, $3$, $4$, and $5$ year olds), school grades, or years of professional experience. The design is inherently repeated measures because the same individual is measured at each chronological stage.

Advantages of Repeated Measures Designs: Statistical Power and Efficiency

• Statistical Power: This is the ability to correctly reject a false null hypothesis. Repeated measures designs are significantly more powerful than between-groups designs, an effect that is magnified as more levels of the IV are added.
• Power increases through two primary mechanisms:     * Reduction of Individual Variability: Because the same participants are used across all conditions, the differences caused by specific individual traits remain constant. This allows researchers to measure and eliminate individual variability from the error term ($ext{residual error}$). By removing this random error, the statistical test becomes more sensitive. This partitioning of variance is central to how the repeated measures ANOVA calculates the $F$ statistic.     * Increased Observations: Obtaining multiple scores from the same participant increases the total number of observations, which helps balance out random error and fluctuations (e.g., environmental noise or temporary distractions).
• Efficiency and Recruitment: These designs requires significantly fewer participants. For example, to achieve a power level of $0.56$ in a specific experiment, a repeated measures design might require only $20$ participants total, whereas a between-groups design would require $74$ participants. This reduces the time spent on recruitment, setting up experiments, and providing instructions.

Case Study: G*Power Analysis and Background Noise Experiment

• GPower Comparison Analysis:      Parameters: T-test ($2$ levels), Effect Size ($d = 0.5$ or Cohen's medium effect), Alpha Level ($ext{Alpha} = 0.05$), Sample Size ($N = 20$).     * Between-Groups Results (with $20$ participants per group): Power is approximately $0.3379$ (rounded to $0.34$).     * Repeated Measures Results (with $20$ participants total): Power increases to approximately $0.56$.
• Cognitive Performance and Background Noise Experiment:     * Layout: Participants performed a cognitive task measuring errors (lower scores indicate better performance) under five conditions: $1. ext{No Sound}$, $2. ext{Repeated Tone}$, $3. ext{Non-speech Music}$, $4. ext{Reverse Speech}$, and $5. ext{Forward Speech}$.     * Between-Groups Analysis (with $10$ participants per group, $50$ total): Resulted in an F-stat of $F = 5.194$ and an effect size of $ext{eta squared} = 0.539$.     * Repeated Measures Analysis (with $10$ participants total): Resulted in a significantly higher F-stat of $F = 41.177$, with a higher partial eta squared.
• Understanding the Correlation: In the noise experiment, individual differences in intelligence, executive functioning, and attention span caused consistent scoring patterns.     * Some participants had a mean error score of $4$, while others had a mean error score of $7.6$.     * Significant correlations exist between levels; for instance, the correlation between the No Sound and Repeated Tone conditions was $r = 0.744$. The lowest correlation recorded was $r = 0.48$. These high correlations allow the ANOVA to be more powerful by accounting for individual consistency.

Further Advantages and Psychological Utility

• Unique Research Questions: Some topics can only be answered via repeated measures, specifically those tracking internal changes over time such as learning, practice effects, or developmental milestones.
• State Manipulations: Studying how different moods or state changes influence the same individual across time.
• Perceptual Research: Examining how perceptual thresholds and sensitivities shift within the same individual.
• High Individual Variability Constructs: Some psychological constructs show massive individual differences at baseline, making repeated measures the preferred design to control for this variance.

Disadvantages and Threats to Internal Validity

• Order Effects: A major threat where the sequence of conditions influences participant behavior or responses systematically. This is often referred to as carryover effects.
• Fatigue Effects: A type of order effect where performance declines in later conditions due to physical or mental exhaustion, boredom, or reduced motivation. Control strategies include rest periods, keeping experiments brief, and providing monetary incentives.
• Practice and Familiarity: Participants may perform better in later conditions simply because they have become acclimated to the task, have lower anxiety, or understand instructions better.
• Sensitization: Participants may discover the experiment's hypothesis over time and develop response biases or specific strategies. They may become better at identifying stimulus cues or shift their standards of judgment.
• Maturation: Biological or psychological changes occurring within the participant during the study, especially problematic for children, adolescents, or the elderly.
• History: External events occurring during the study period (days, weeks, or years) that affect the dependent variable.
• Instrumentation: Changes in how an instrument scores data over time, independent of the independent variable, often occurring if data is scored inconsistently across measurement periods.
• Attrition: The loss of participants due to withdrawal, moving away, or boredom. This creates bias because those who drop out may differ fundamentally from those who persist.

Control Strategies for Order Effects

• Randomization: Randomizing the order in which conditions are presented.
• Full Counterbalancing: Presenting every possible unique sequence of conditions to participants. For $3$ conditions ($A$, $B$, and $C$), there are $6$ unique sequences ($ABC$, $ACB$, $BAC$, $BCA$, $CAB$, $CBA$). This requires participant totals in multiples of $6$. It becomes impractical with $4$ or more levels.
• Latin Square Design: A strategy for designs with many levels ($4+$) ensuring each condition appears once in each ordinal position and each condition precedes and follows every other condition equally often.
• Analysis Checks: Including order as a factor in the final data analysis to see if results differ based on condition sequence.
• Reducing Sensitization Bias: Using placebo conditions, subtle manipulations, deception, or filler tasks to mask the true purpose of the experiment.