12 THREATS TO INTERNAL VALIDITY

exam 2

Maturation

An experimental group improves over time only because of natural development or spontaneous improvement.

prevention - comparison group that can reveal whether there is an effect of the IV above and beyond any effect of maturation.

History

An experimental group changes over time because of an external factor that affects all or most members of the group.

Prevention -comparison group that can reveal whether there is an effect of the IV above and beyond any effect of history.

Regression to the mean

An experimental group whose average is extremely high or low at the pretest will get better or worse over time because the random events that caused the extreme pretest scores do not recur the same way at the posttest.

prevention - comparison group and careful inspection of the pattern of results.

regression effect only possible if lines don’t overlap, if they overlap - clear effect.

Attrition

An experiment group changes over time, but only because the most extreme cases have systemically dropped out and their scores are not included in the posttest.

prevention - remove pretest data for participants who dropped out.

Testing

A type of order effect: an experimental group changes over time because repeated testing has affected the participants. Practice effects (fatigue/boredom) is a subtype.

prevention - add comparison group/posttest only/if pre/post use different versions of the test

Instrumentation

An experimental group changes over time, but only because the measurement instrument (testers count too) has changed.

prevention

• if 2 different tests are used make sure they’re calibrated

• use clear coding manuals to retrain coders throughout

• counterbalance versions of the test

• post-test only design

Observer bias

An experimental group’s ratings differ from a comparison group’s but only because the researcher expects the groups’ ratings to differ.

prevention

• conduct a double-blind study in which neither the participants nor the researchers who evaluate them know who is in treatment and comparison groups

  OR

• conduct a masked design where participants know what group they are in but observers do not

Demand characteristic

Participants guess what the study’s purpose is and change their behavior in the expected direction.

Placebo effect

Participants in an experimental group improve only because they believe in the efficacy of the therapy or drug they receive.

design confound

a second variable that unintentionally varies systematically with the IV

selection effect

in an independent-groups design, when the two independent variable groups have systematically different kinds of participants in them.

order effect

in a repeated-measures design, when the effect of the independent variable is confounded with carryover from one level to the other or with practice, fatigue, or boredom.

selection-history threat

an outside event or factor systematically affects participants at one level of the IV

ex: go green study - dorm used as a study group was undergoing construction, therefore using more energy

selection-attrition threat

Participants in only one experimental group experienced attrition

ex: depression study, most depressed dropped out because treatment was particularly hard on them.