Threats to External Validity

Effects Differ by Group

Example: “Double-dose” algebra leads to increases in high school graduation rates in the Chicago Public Schools. Would the same thing happen if we gave this intervention to the highest-achieving children? What about 3rd graders?


Solution: You cannot assume that just because it worked for one group, it will work for another group. Be careful how far you can generalize, and collect more data if you need to.

Effects Differ Across Settings

Example: Grocery store signs intervention increased parent-child talk by 33% only in low-income stores.


Solution: An intervention can work in one setting, but not in another. Do the experiment again but in a different setting. Be careful how far you can generalize, and collect more data if you need to.

Effects Differ Across Intervention

Example: long vs. short interventions; major vs. minor life stressors; trained vs. untrained teachers

Solution: If your experiment is conducted over a year, will it work again if you do it over a month? Be careful how far you can generalize, and collect more data if you need to