2.1| Validity, Types of Validity, Threats to Internal Validity

Low validity – the test doesn’t measure the intended concept (general validity).

A researcher uses a spelling quiz to measure children's reading comprehension.

Low validity – outcome doesn't reflect what the study intends to measure (general validity).

A study claims a therapy reduces depression but only measures sleep hours.

Low internal validity – due to a confounding variable.

A study finds students improve after using a learning app, but they also received tutoring.

High internal validity – random assignment reduces bias.

Participants are randomly assigned to groups in a drug effectiveness study.

Low population validity – limited generalization to other groups.

A study on stress is conducted only on medical students from one school.

Low ecological validity – the setting doesn’t reflect real-life conditions.

An experiment in a quiet lab is used to predict behavior in a noisy workplace.

Low construct validity – the measure doesn’t fully capture the concept.

A survey claims to measure depression but only asks about diet and sleep.

High construct validity – tool accurately represents the psychological concept.

Researchers use a widely accepted scale to measure self-esteem.

Low temporal validity – outdated findings.

A 1995 study on technology use is used to make conclusions about Gen Z.

Low temporal validity – results don’t hold over time.

A study on social media habits during the pandemic is repeated in 2025 and shows different results.

Confirmation Bias

A teacher who believes that girls perform better in math tends to give more praise and attention to female students during math lessons, while overlooking the achievements of male students.

Sampling Bias

A researcher studies the popularity of a new fitness trend by surveying gym-goers at a high-end fitness center.

Experimenter Bias

A scientist conducting an experiment on the effectiveness of a new drug unconsciously encourages participants who are taking the drug to report improvements, while downplaying the feedback of those who experience no change.

Selection Bias

A researcher is studying the effects of a new diet on weight loss. They recruit participants by posting flyers in a local gym, assuming that all participants are already health-conscious. As a result, the sample is biased because it does not represent people who may not go to the gym or those with different health behaviors.

History Bias

A study is tracking the effects of school stress on children’s health. During the study, a significant earthquake occurs, causing heightened stress among the participants. The external event (the earthquake) could influence the results of the study, making it harder to isolate the effects of school stress alone.

Maturation Bias

A study is examining the effectiveness of a new educational program on children’s reading skills over a year. However, during that time, the children naturally mature and develop their reading skills, making it difficult to know whether improvements are due to the program or just natural growth.

Testing Effect

In an experiment on problem-solving skills, participants are given the same test three times. Over time, they become more familiar with the questions and start answering them differently, not because of an actual improvement in skills but because of their familiarity with the test format.

Instrumentation Bias

A researcher is conducting a study on stress levels and uses a heart rate monitor to measure physiological responses. Halfway through the study, the heart rate monitor breaks, and they use a different brand, which measures heart rate slightly differently. This introduces variability in the measurement tool, affecting the results.

Regression to the Mean

A researcher selects students who performed extremely poorly on a math test to track improvements after a tutoring program. After the program, those students’ scores improve, but this could just be due to the natural tendency of extreme scores to move closer to the average, rather than the tutoring itself.

Experimental Mortality

In a long-term study of elderly patients taking a new medication for memory loss, some participants drop out because they experience side effects. This could lead to biased results because the participants who remain might be those who didn’t experience negative side effects, making the medication appear more effective than it actually is.

Demand Characteristics

In an experiment on prosocial behavior, participants know they are being observed while helping others. As a result, they behave more altruistically because they believe that’s what the researcher expects.

Experimenter’s Bias

A researcher is testing a new treatment for anxiety. The researcher is very confident that the treatment will work and subconsciously gives more positive feedback and encouragement to participants who are receiving the treatment. This subtle influence could lead to skewed results.