PY2107 pp.186-195: Internal Validity

Internal validity

The correctness of inferences made by researchers about cause and effect

Confounding

Occurs when an extraneous variable co-occurs with the independent variable and affects the dependent variable

Confounding extraneous variable

An extraneous variable that co-ccours with the independent variable and affects the dependent variable

An extraneous variable that systematically varies with the independent variable and causes changes in the dependent variable

Threats to internal validity

History

• Basic history

• Differential history

Maturation

Instrumentation

Testing

• Testing effect

Regression artifacts/regression toward the mean

Attrition

• Differential attrition

Selection

Additive and Interactive Effects

• Selection-history

• Selection-maturation

• Selection-instrumentation

• Selection-testing

• Selection-regression artifact

Best solution: construct the groups at the beginning of your experiment using random assignment

Constancy

The influence of an extraneous variable is same on all of the independent variable groups

Equating the groups

Using control strategies to make the influence of extraneous variables constant across the independent variable groups so that the only systematic difference between the groups is due to the influence of the independent variable

Control

Exerting a constant influence (achieving constancy)

History

Any event that can produce the outcome, other than the treatment condition, that occurs during the study before posttest measurement

Treatment and event would both occur between the pre-test and posttest, and you would not know if a pretest-to-posttest improvement was due to the treatment or history event because you cannot separate their influences

Not a problem when a control group is added as long as the event affects both groups (but will not fix differential)

Greater risk the longer the time lapse of a research study is

Differential history

The groups in a multigroup design experience different history events that result in differences on the dependent variable

When one group experiences the history event, but the other does not

Maturation

Any physical or mental change that occurs with the passage of time and affects dependent variable scores

Not related to specific external events but reside from within the individual

To control, include a control group that does not receive treatment

Instrumentation

Changes from pretest to posttest in the assessment or measurement of the dependent variable (for designs lacking a control group)

Measurement using human observers most risk:

• Use more than one observer

• Have each observer go through a training program

• Data collected must coincide

Testing effect

Changes in a person’s score on the second administration of a test resulting from having previously taken the test

• Sensitise pp to the test topic/related issues

• Gives pp practice with taking the test, familiarises you with content

Problematic for one-group design, but adding a control group usually eliminates this threat

Regression artifacts/regression toward the mean

Effects that appear to be due to the treatment but are due to regression to the mean

Operates when pps are selected based on extreme (high or low) scores

• Pps with very high scores tend to decrease

• Pps with very low scores tend to increase without any treatment from pretest to posttest measurement

Control: two-group design: even if regression occurs, the difference between the groups will not be due to a regression artifact

Attrition

Loss of pps because they don’t show up or they drop out of the research study

Differential attrition

In a multigroup design, groups become different on an extraneous variable because of differences in the loss of pps across the groups

Not an internal validity problem in one-group design, but will negatively affect generalisability

Selection

Production of non-equivalent groups because a different selection procedure operates across the groups

Random assignment is th best procedure for equating groups

Additive and interactive effects

Differences between groups is produced because of the combined effect of two or more threats to internal validity

Selection-history

The groups are exposed to the same history event, but they react differently because they were not equated

Selection-maturation

The groups undergo different rates of maturation because they were not equated

Selection-instrumentation

The groups react to changes in instrumentation differently because they were not equated

Selection-testing

The groups react to the pretest differently, because they were not equated

Selection-regression artifact

The groups show different amounts of regression to the mean, because they were not equated