Chapter 11: More on Experiments; Confounding and Obscuring Variables

Design confounds

some other variable systematically varies with the independent variable

selection effects

groups that should only differ based on levels of the IV have systematically difference types of participants

order effects

order in which levels of the IV were presented affects the DV

maturation threat

a change in behaviour that emerges because time is generally passing

history threat

a change in behaviour due to a specific event that affects the experimental group members at the same time as the treatment group

regression threat/regression to the mean

extremely high performance at time 1 is likely to be less extreme at time 2 (bc we've started at an extreme, we can't stay at that extreme over time— only occurs in pretest posttest)

attrition threat

as time goes on, people drop out of the study (only issue if systematic)

testing threat

type of order effect where there is a change in participants as a result of experiencing the DV more than once (practice effect & fatigue effect)

practice effect

scores go up due to practice

fatigue effect

scores go down due to fatigue

instrumentation threat/instrumentation decay

when a measuring instrument changes over time ( can be phys change in tool or change in the application of observation criteria)

Selection-history threat (combined threat)

outside event or factor systematically affects participants in one group, but not the other

selection-attrition threat (combined threat)

participants in only one group experience attrition

observer bias

researchers expectations influence their interpretation of the data

observer effects

researchers expectations influence the behaviour of the participants

demand characteristics

participants figure out what the study is about and behave in the way that they should

placebo effects

participants' expectations influence their behaviour

double-blind study

neither participants nor observers know who is in the treatment of control group

masked design/ single blind design

participants know which group they're in but observers don't

double-blind placebo control study

neither participants nor observers know who is in which group, specifically because a special comparison group is used that is receiving the placebo therapy

null effects/results

non-significant results (no association, regression, relationship)

3 reasons for null effects/results

1- lack of betweens group difference (different individuals in different manipulations; ab levels)

2- within-groups variability obscured the group differences (individuals themselves; ab participants)

3- there's no significant difference

weak manipulations (between-groups difference)

the difference between levels of the IV are too small to make a difference; haven't done a good job manipulating our variables

insensitive measures (between-groups diff)

DV is not operationalized with enough sensitivity; such large ranges

ceiling effect (between-groups effect)

participants' scores on the DV are clustered at the high end

floor effects (between-groups effect)

participants' scores on the DV are clustered at the low end

design confounds acting in reverse (rare case)

design confounds counteracting the true effect of an IV

manipulation check

adding a second DV that should be impacted by the IV, to make sure the IV manipulation worked

measurement error (variability within-groups)

a human/instrument factor that can change a person's true score on the DV randomly

solutions to measurement errors

- use reliable, precise measurements

- measure more instances

individual differences (variability within-groups)

participants differ in their scores across variables, including those related to the experiment

solutions to individual differences

- within-groups & matched groups design (repeated measures)

- larger sample

power

likelihood that a study will yield a statistically significant result when the IV really has an effect on our DV (less variability & larger sample > power)

situation noise (variability within-groups)

any kind of external distraction that could cause variability within groups that obscures the data; usually environmental forces

solution for situation noise

controlling the surroundings of an experiment