Confounds/Artifacts

confound

variable that systematically varies with the IV and provides an alternative explanation for changes in DV

• 3rd variables

• ex: link between coffee/stress level confounded by poor sleep

pseudoexperiment

exposing people to an IV and then seeing if there are any outcome changes in the DV

• creates damaging/false links when they don’t actually exist

• ex: MMR vaccine → autism

maturation

systematic changes that occur spontaneously over time not due to variables being studied

• ex: kids naturally grow and develop over time

• sometimes fixed w/ comparison group

history

something specific occurs during the study period that affects everyone at same time as IV

• ex: new anxiety medication + pandemic = unsure if unchanged anxiety levels due to med failing or pandemic overwhelming med’s effects

testing/practice effects

when taking a test/completing a measure previously influences later performance

• fixed by comparison groups, excluding pretest scores, time delays between tests

instrumentation

when the way a variable is assessed changes over time within a study

• ex: paper test → online format

• ex: observational criteria shifts as experience increases

regression to the mean

tendency for extreme scores to become closer to average when measured again within a study

• difference between true and observed score

• ex: coin flip 10 times and get 9 heads, but on next 10 flips, result naturally balances out closer to 5 heads

• fixed by comparison groups or checking between-group differences for similar groups

attrition

systematic drop out when participants leave a study before finishing

• ex: PTSD drug trial looks effective because of low anxiety reports at the end, but patients with bad side effects already dropped out

• fixed by adjusting study design

assignment bias

when assignment leads to systematically different groups prior to manipulation inside a study

• ex: separating students based on intelligence makes it hard to see if tutoring is truly effective

• fixed by random assignment

artifacts

variable held constant in a study/studies but might define the context in which any observed IV/DV relationship occurs

• affects ability to generalize

• ex: memory trick works in silent research room, but fails in loud classroom

participant expectancies

purposely behaving in ways consistent with experimenters’ hypotheses

• demand characteristics

participant reactance

purposely behaving in ways inconsistent with experimenters’ hypotheses

social desirability/evaluation apprehension

behaving in ways to appear socially desirable

acquiescence

participant inclination to say yes, even if it’s not how we actually feel

addressing participant bias

cover stories, unobtrusive observations, multiple questions asked in different ways, indirect measures

addressing experimenter bias

blind/double blind studies, standardized instructions

threats to generalizability across people

selection bias, homogenous samples, self-selection effects, WEIRD samples

threats to generalizability across research

fatigue effects, practice effects, experimenter characteristics, measurement time/operationalization

internal/external validity relationship

inverse relationship; must give up control for generalizability and vice versa