Special Research Designs I-II

What makes a study quasi-experimental?

• No random assignment

• Uses pre-existing or self-selected groups

• IV is “manipulated,” but researchers cannot fully control who enters each level

• Has many but not all elements of a true experiment

How do quasi-experiments differ from correlational designs?

Both:

• No random assignment

• Only measure outcome variables

Quasi:

• Researcher attempts to manipulate an IV (but lacks full control)

Correlational:

• No manipulation at all; uses predictors, not IVs

Why might a true experiment not be possible?

• Ethical issues (e.g., illness, drug use)

• Impossible/unrealistic to manipulate (e.g., age, sex, genetics)

What is the structure of a one-group post-test only design?

Treatment → Post-test only (no baseline)

What are limitations of one-group post-test only design (e.g., VapeAway study)?

• No baseline/pre-test → cannot see change

• No control group → cannot attribute effect to treatment

• Cannot rule out alternative explanations (history, maturation)

• Cannot claim causation

What is the one-group pre-test/post-test design?

Pre-test → Treatment → Post-test

What are the limitations of one-group pre-test/post-test design?

• Still no control/placebo group

• Threats to internal validity (history, testing effects)

• Cannot ensure the treatment caused the change

What is a non-equivalent group post-test only design?

• Two groups formed based on existing differences (self-selection, natural groups)

• Both measured only at post-test

• Between-groups quasi-experimental design

What are the key limitations of non-equivalent group post-test only design?

No random assignment → selection bias

• Cannot claim causality

• No baseline (cannot measure change)

• Alternative explanations uncontrolled (motivation, time, difficulty to quit, etc.)

What is a non-equivalent group pre-test/post-test design?

Both groups: Pre-test → Post-test
Still no random assignment (thus selection bias remains)

What are the main limitations of non-equivalent group pre-test/post-test design?

• Improved design (has a baseline), but…

• Still lacks random assignment → cannot fully claim causation

• Internal validity remains threatened

Can you use t-tests and ANOVA with quasi-experiments?

Yes, same statistical techniques can be used —
BUT interpretations must reflect the study design and lack of full control.

What are the three requirements for causality?

• Covariation of cause and effect

• Temporal precedence (cause before effect)

• Eliminate alternative explanations

Hypothesis: Having multiple concussions impacts cognitive functioning.

1. Recruit 50 American Football players who’ve been diagnosed with a concussion

2. Measure their cognitive function using Raven’s Matrices

3. Results - accuracy on Raven’s Matrices = 20%

4. Conclusion: Concussions lead to impaired cognitive functioning

Why does the football concussion study fail?

• No covariation (no comparison group)

• No temporal precedence (only one time point)

• Cannot rule out alternative causes

• Overall: extremely weak design for causal inference

What is an interrupted time series (ITS) design?

• Multiple pre-tests and post-tests around a “natural manipulation”

• Looks for a shift in trend after the intervention event (e.g., EBT introduction)

Strengths of ITS (interrupted time series)?

• Stronger internal validity than single pre-post

• Can detect trends over time

• Good for natural interventions

Limitations of ITS (interrupted time series)?

• Potential confounds (e.g., crime decreasing linearly anyway)

• History effects at the same time as the intervention

What is a control series design?

• ITS (interrupted time series) with a non-equivalent control group also tracked over time

• Compares treatment trend vs. control trend

How do you interpret results if both groups follow the same trend?

• Suggests the intervention likely did not have a significant effect

• Trend may be driven by broader societal changes, not the treatment

Why are developmental studies quasi-experiments?

Because age cannot be randomly assigned → no manipulation and no randomization.

What is a cross-sectional design?

• Different age groups measured at one time

• Useful for observing group differences and cohort effects

• Fast and cheaper
  Limitations: cohort effects, cannot show individual change over time

What is a longitudinal design?

• Same individuals followed over time

• Higher internal validity

• Can track individual developmental trajectories
  Limitations: time-consuming, expensive, attrition, history effects

What are sequential designs?

• Combines cross-sectional + longitudinal

• Multiple cohorts followed over time

• Best for separating age, cohort, and time-of-measurement effects
  Limitations: very resource-intensive, attrition, tech/time changes

What is p-hacking?

Using unethical researcher practices to artificially obtain statistically significant results.

Examples of p-hacking?

• Collecting extra data until significant

• Dropping outliers to support your hypothesis

• Using several measures but reporting only the significant one

• Reporting exploratory significant interactions as if predicted

• b) is correct

• We cannot ethically manipulate who vapes and not by random assignment

• b) is correct

• Not ethical

• We need to make assumptions → e.g., pre-test data can be found in past records; we can take pre-existing data (key strength of quasi-experimental design)

• We can just collect and compare → nothing needs to be manipulated

• d) is correct

Educational level → person variables are not easily manipulated

a) is correct

To control for football players, they can be compared with normal people with concussion

a) is correct

c) is correct