Non-Randomized Quasi-Experimental Designs (QED)

Comprehensive review flashcards covering non-randomized quasi-experimental designs (QED), causation requirements, notation symbols, validity threats, selection-maturation patterns, and empirical study examples.

What are the three basic requirements for establishing causal inference in quasi-experimental designs?

1. Cause precedes effects, 2. Cause covaries with effects, and 3. Alternative explanations are implausible.

How is internal validity defined in the context of research design?

"Did in fact the experimental stimulus make some significant difference in the specific instance".

What are the three primary principles used to address causation in quasi-experimental designs (QED)?

1. Identification of plausible threats to internal validity (how they operate within a given design), 2. Control by Design (adding design elements to rule out confounding or validity threats), and 3. Coherent Pattern Matching (ensuring predictions match the observed pattern of effects with no alternative explanation).

What is the threat of ambiguous temporal precedence?

A threat occurring when it is unclear whether the cause preceded the effect, common in cross-sectional correlational studies where bidirectional (reciprocal) effects cannot be ruled out.

What is the threat of selection in quasi-experimental designs?

Systematic differences across conditions in respondent characteristics due to volunteering or self-selection rather than random assignment, which violates the counterfactual.

How do the internal validity threats of history and maturation differ?

History refers to extraneous events occurring concurrently with the treatment (local secular trends), whereas maturation refers to naturally occurring internal changes over time (such as growing older, becoming more experienced or intelligent, or experiencing age-related functional declines).

What are regression artifacts (regression to the mean)?

A phenomenon where units selected for extreme scores on a pretest naturally produce less extreme scores upon subsequent testing or measurement, often mistaken for a treatment effect or spontaneous remission.

How do testing threats differ from instrumentation threats?

Testing refers to changes caused by participants' exposure to a test influencing subsequent administrations, whereas instrumentation refers to changes in the measurement tool itself over time (such as equipment changes or shifts in the construct definition).

What do the design notation symbols $X$, $\not{X}$, $O_{\#}$, and $NR$ represent?

$X$ represents treatment manipulation, $\not{X}$ represents treatment removal, $O_{\#}$ represents an observation occasion (e.g., $O_1$ is the first observation), and $NR$ (separated by a dashed line) represents non-random assignment.

Why is the one-group post-test-only design ($X\ O_1$) rarely used, and which validity threat is absent?

It is rarely used because the lack of a pre-test leaves change uncertain and the lack of a control leaves the counterfactual unknown, allowing almost all threats to operate; however, ambiguous temporal precedence does not operate because the treatment clearly precedes the post-test.

What is the key advantage of a double-pretest design ($O_1\ O_2\ X\ O_3$)?

It helps identify threats from maturation, regression, and history by observing baseline trajectory between $O_1$ and $O_2$ and verifying whether the change between $O_2$ and $O_3$ exceeds that baseline change.

What is a nonequivalent dependent variable ($O_{\#B}$) and what role does it serve?

A dependent variable measuring a similar construct that is predicted not to change as a result of the treatment, but is expected to respond to the same internal validity threats and confounders as the primary outcome ($O_{\#A}$).

In this removal treatment design graph ($O_1\ X\ O_2\ O_3\ \not{X}\ O_4$), what distinguishes the interpretable outcome from the uninterpretable outcome?

An interpretable outcome shows a sharp decline back toward baseline following removal ($\not{X}$), demonstrating that the treatment effect dissipates; an uninterpretable outcome shows sustained elevated performance, which could reflect lingering treatment effects or unaddressed maturation.

What is a proxy pretest design, and what biases can it detect?

A design ($NR\ O_{1A}\ X\ O_{2B} \,/\, NR\ O_{1A}\ O_{2B}$) that collects pretest data on a variable conceptually related to the outcome and correlated with the posttest instead being the outcome itself, allowing detection of selection and attrition bias.

Which pattern of selection-maturation threat is illustrated in this diagram?

Both groups growing apart in the same direction, also known as the fan spread model of maturation.

What selection-maturation pattern is depicted in this pre-to-post comparison graph?

Outcomes that cross-over in the direction of relationships (crossover effect).

How does a switching replications (cross-over) design work?

One group receives treatment first while the second serves as a control ($NR\ O_1\ X\ O_2\ O_3$), after which the control group receives the treatment ($NR\ O_1\ O_2\ X\ O_3$), assuming the initial treatment has no lingering impact.