Introduction to Simple Experiments and the Four Validities

Vocabulary practice flashcards reviewing experimental design, validity types, internal validity threats, and interrogation of null effects based on psychology research methods notes.

Four Validities for Causal Claims

The four criteria used to evaluate causal claims: Construct validity, External validity, Statistical validity, and Internal validity.

Control Variable

Any variable that an experimenter holds constant on purpose (such as serving the exact same type of pasta to all participants).

Covariance

A required criterion for establishing causation showing that variable $A$ and variable $B$ are related to one another.

Temporal Precedence

A criterion for causation establishing that the causal variable ($A$) came first in time before the effect variable ($B$).

Internal Validity

The degree to which an experiment ensures there are no alternative explanations or confounds responsible for the observed outcome.

Treatment Group

The condition or conditions in an experiment that receive the active level of the independent variable.

Control Group

A condition in an experiment that represents a neutral or no-treatment baseline.

Placebo Group

A control group that receives an inactive treatment which appears active to the participants.

Systematic Variability

Variability in an experiment that changes consistently with the levels of the independent variable, creating a design confound.

Unsystematic Variability

Random variability that does not change consistently with the independent variable and is not a confound, though excessive amounts can obscure group differences.

Selection Effect

A threat to internal validity in independent-groups designs where participants in one condition are systematically different from those in another condition.

Random Assignment

The practice of assigning participants to experimental conditions randomly to eliminate systematic individual differences and prevent selection effects.

Matched Groups

An assignment method where participants are sorted on a specific variable (such as GPA) and then randomly assigned across conditions in equal pairs or clusters.

Independent-Groups Design

An experimental design (also known as a between-subjects design) where different groups of participants are exposed to only one level of the independent variable.

Within-Groups Design

An experimental design (also known as a within-subjects design) where each participant is exposed to all levels of the independent variable.

Posttest-Only Design

An independent-groups experimental design in which the dependent variable is measured only once, after exposure to the independent variable.

Pretest/Posttest Design

An experimental design where the dependent variable is measured twice: once before exposure to the independent variable and once after.

Repeated-Measures Design

A type of within-groups design in which participants are measured on the dependent variable sequentially after exposure to each level of the independent variable.

Concurrent-Measures Design

A type of within-groups design in which participants are exposed to all levels of the independent variable at the exact same time.

Order Effect

A threat to internal validity in within-groups designs where exposure to one condition alters how participants respond to subsequent conditions.

Practice Effects

A type of order effect where participant performance improves due to practice or deteriorates due to fatigue over repeated testing.

Carryover Effects

A type of order effect where the influence of one condition lingers or contaminates the participant's response in a subsequent condition.

Counterbalancing

A technique used to control for order effects in within-groups designs by presenting experimental conditions to participants in different sequences.

Full Counterbalancing

A counterbalancing method in which all possible condition orders are represented equally across participant groups.

Partial Counterbalancing

A counterbalancing method in which only a subset of all possible condition orders is used, ensuring conditions appear in each position at least once.

Construct Validity

An evaluation of how well the dependent variables were measured and how effectively the independent variables were manipulated.

Manipulation Check

An extra measure inserted into an experiment to verify that the independent variable produced the intended change in participants.

Pilot Study

A small-scale, mini-version of a study conducted prior to the main experiment to test the effectiveness and feasibility of manipulations.

External Validity

The extent to which causal claims and experimental results generalize to other populations or situations.

Statistical Validity

The evaluation of how precise and strong the statistical findings are, including effect size ($d = 0.2$ for small, $d = 0.5$ for moderate, $d = 0.8$ for large) and confidence intervals.

Maturation Threat

A threat to internal validity in pretest/posttest designs where behavioral change occurs spontaneously and naturally over time rather than from the intervention.

History Threat

A threat to internal validity occurring when external events outside the study systematically affect most participants between pretest and posttest.

Regression Threat

A threat to internal validity where extreme pretest scores naturally move closer to the average (mean) upon retesting.

Attrition Threat

A threat to internal validity occurring when systematic dropout of extreme participants (highest or lowest scoring) skews posttest results.

Testing Threat

A threat to internal validity where taking a pretest measure alters how participants respond on a subsequent posttest.

Instrumentation Threat

A threat to internal validity that occurs when a measuring tool, apparatus, or observer rating criterion changes over time.

Observer Bias

A threat to internal validity occurring when researchers' expectations unintentionally influence their observations or interpretation of participant data.

Demand Characteristics

Cues in an experiment that lead participants to guess the study hypothesis or expectations and alter their behavior accordingly.

Placebo Effect

An improvement in participant outcomes caused strictly by the belief that they are receiving an active treatment when given an inactive one.

Double-Blind Study

An experimental procedure in which neither the participants nor the researchers observing them know who is assigned to which experimental condition.

Null Effect

A finding that the independent variable did not produce a statistically significant difference in the dependent variable.

Ceiling Effect

A problem where scores cluster at the high end of a scale because the dependent variable measure is too easy or has an upper limit.

Floor Effect

A problem where scores cluster at the low end of a scale because the dependent variable measure is too difficult or has a lower limit.

Measurement Error

Factors that artificially inflate or deflate a participant's true score on a dependent variable due to measurement inaccuracies.

Situation Noise

External environmental distractions that cause unsystematic variability within groups and obscure true between-group differences.

Power

The likelihood that a study will yield a statistically significant result when the independent variable truly has an effect.

Publication Bias

Also known as the file drawer problem; the tendency for journals to publish positive or significant findings more frequently than null or non-significant results.

Interaction Effect

The effect that occurs when the impact of one independent variable on the dependent variable differs depending on the level of another independent variable, indicating that the variables work together in a way that is not simply additive.

Crossover Interaction

“It depends”

A type of interaction where the effect of one independent variable on a dependent variable reverses direction at different levels of a second independent variable. This results in a crossover of the lines representing the different conditions in a graph. (Ex. Introverts line sloping downwards and Extroverts line slopes upwards where test performance is measured on the Y axis and Noise on the X axis)

Spreading Interaction

“Only When”

An independent variable has a strong effect at one level of a second independent variable, but a weaker effect or no effect at another level, it avoids misleading main effects, and allows for actionable insight. (Ex. In group studying only effective when there is low baseline knowledge.)

Factorial Design

Design in which there are two or more independent variables (Ex. age and cell phone use effect on onset breaking time)

What does it mean when the second IV did not moderate the first IV?

There was no interaction effective. (Ex. In the study assessing the effect that phone use and age has on break reaction time, age did not moderate the cell phone use condition since in both age ranges reaction time went up once cell phone use was applied)

Independent-Groups Factorial Design, how many groups?

Both IVs are studies as independent groups, so a 2 X 2 design would have 4 groups (25 in each group = 100)

Within-Groups Factorial Design, how many groups?

Both IVs are manipulated within groups, only one group for all four cells (25 in a group means 25 total)

Mixed Factorial Design, how many groups?

One IV is manipulated as independent-groups and the other as within group (pre-test vs. post-test). For 25 in a group there would be 50 total in a 2 X 2 design. Because 25 would be getting active learning/easy and difficult content then another 25 would be getting inactive learning/easy and difficult content.

There is also 2 X 3 ans 3 X 4

Three IV’s

3 main effects, additionally one three way interaction of the three variables