Validity in Research Methods

11 question-and-answer flashcards covering definitions, types, and threats to validity in research.

In research methods, what does the term "validity" mean?

The extent to which a formal measurement tool actually measures what it is intended to measure and the extent to which conclusions and actions based on that measurement are appropriate and accurate.

What are the four major types of experimental validity identified in the lecture?

Internal validity, construct validity, external validity, and statistical conclusion validity.

What central question does internal validity ask about an experiment?

Whether the observed differences in the dependent variable were caused by manipulation of the independent variable rather than by some extraneous factor, allowing a true causal inference.

Give an example of a “history” threat to internal validity.

Any event occurring between the pre-test and post-test besides the treatment—e.g., an economic recession, war, or school closure due to COVID-19—that could influence participants’ scores and mimic a treatment effect.

What is the statistical regression (regression to the mean) threat and when is it likely to appear?

It is the tendency for extreme scores on an initial measurement to move closer to the average on a second measurement when the measure is less than perfectly reliable; it often appears when groups are chosen for very high or very low initial scores.

Why is selection considered a threat to internal validity, and how does it arise?

Because outcome differences may stem from pre-existing group differences rather than the treatment; it arises when participants are non-randomly assigned, producing unequal groups from the outset.

Which experimental design practice is recommended to control threats like selection and selection-by-interaction?

Random assignment of participants (and/or of experimental conditions) to the different groups.

What does construct validity evaluate in a study?

Whether the operational definitions used truly represent the theoretical constructs and whether the observed causal link can be generalised from the measured variables to the underlying concepts.

According to the lecture, how can you distinguish between an artifact and a confound?

An artifact is an alternative explanation that can be easily isolated and manipulated independently of the main independent variable, whereas a confound is an inseparable component of the manipulation that offers an alternative theoretical explanation (e.g., experimenter or participant expectations).

Define external validity and cite one specific interaction that can threaten it.

External validity is the ability to generalise research findings across populations, settings, times and situations; one threat is the interaction of selection and treatment, where results apply only to participants who were initially motivated, skilled, or otherwise unrepresentative.

In statistical conclusion validity, what are Type I and Type II errors?

A Type I error is falsely concluding that a relationship exists when it does not (false positive), whereas a Type II error is failing to detect a real relationship (false negative).