Internal Validity

Internal validity assesses the quality of the experimental design in supporting a causal claim.
It focuses on whether the experiment was well-designed to avoid confounds.
A well-designed experiment should lead to only one possible claim.
A confound arises when there are alternative explanations for the results, undermining the claim.
Internal validity contrasts with external validity.
External validity examines whether the experimental results generalize to real-world settings.
Internal validity focuses solely on the design of the experiment itself.
It asks: Does the experiment unambiguously support the researcher's claim, or are there other potential explanations?
Good internal validity means the research design effectively rules out alternative explanations, supporting the claim.

Example: Preschool Quality and College Admissions

The claim: "Better preschools lead to better colleges."
Original claim was an association claim: "People who go to better preschools tend to get into better colleges."
The revised claim is a causal claim, asserting that better preschools cause better college admissions.
The causal claim posits that a good educational environment in preschool positively affects later college prospects.
Private preschools can cost $30,000-$40,000 per year and require consideration if the investment is warranted.
If the causal claim is true, investing in high-quality preschool is critical for long-term educational success.
This suggests that the early years are crucial for building a foundation that extends to college.

Using X and Y variables to illustrate the causal claim:
- X = Better preschool
- Y = Better college
The claim: Better preschool (X) causes better college (Y).
The question: Is it this simple, or are there other factors involved?
Taking for granted there's a relationship: children who go to better preschools eventually tend to get into better colleges.
Is preschool the only reason for those advantages?
Association vs. causation: The association claim may be correct, but not necessarily the causal claim.

Money (Socioeconomic Status)
- Money can be a confound. Families who can afford better preschools might also afford better colleges.
- This suggests that socioeconomic status, rather than preschool quality, is the primary driver.
- More exclusive, private, and resource-rich institutions are generally considered “better.”
Parental Investment in Education
- The level of parental involvement in a child's education also plays a significant role.
- Parents who invest in better preschools may also consistently support and push their children academically.
- Parental involvement throughout a child's education could be the reason for college success rather than preschool quality alone.
These two variables are not mutually exclusive.
Teasing out if it's really the preschool and not these other variables requires complex statistical analysis.

Revisiting the X and Y relationship with potential confounds:
- Better preschool (X) causes better college (Y)? - Questionable.
- Does higher socioeconomic status (Z) cause better college (Y)?
- Does greater parental investment in education (Z) cause better college (Y)?
Z represents a third variable affecting the relationship between X and Y.
It undermines the claim that preschool quality alone leads to better college outcomes.

The third variable, Z, gets in the way of establishing a clear causal relationship between X and Y.
It introduces a confound, confusing the true cause of better college admissions.
We cannot definitively say that preschool quality is the determining factor.
Understanding confounds is critical for evaluating research claims.
If preschool isn't the key factor but money, it may not matter as much which preschool the family chooses.

A third variable is a confound.
Confound undermines our ability to interpret the validity of this statement: Better preschool causes better college.
A confound results in multiple potential causes: is it X or Z that is causing Y?
Third Variable Problem: X \rightarrow Y , but also Z \rightarrow Y.
The possibility of a third variable means we are confused as to which is the cause.
Recognizing confounds is essential for critical thinking and evaluating research.
Skeptical inquiry helps identify these confounding factors.

Good internal validity means potential confounds have been successfully ruled out.
An experiment's design eliminates alternative explanations which makes the causal claim more believable.
Demonstrating internal validity involves designing experiments to rule out alternative factors.
Ruling out confounds is how to find evidence that the claim that better preschools lead to better colleges is internally valid.
It requires controlling for socioeconomic status, parental investment, and other potential influences.
This is more complex than demonstrating an association claim.
To control socioeconomic status:
- Recruit families across different socioeconomic levels.
- Track their progress from preschool to college.
- Use statistical techniques to determine the unique impact of preschool quality, controlling for income.
To measure parental investment:
- Quantify parental involvement (e.g., hours spent on schoolwork).
- Measure the impact on college admissions.
These studies are challenging and time-consuming.
Causal claims require substantial evidence to rule out alternative explanations.

Longitudinal studies are complex; families' circumstances can change over time.
Attrition (participants dropping out) can introduce bias.
Unaccounted variables at home or other situations can affect academic outcomes.
Establishing causal relationships in real-world settings is inherently messy.
Association claims easier to support, showing correlation without causation.
Causal claims demand more robust evidence to support them.
Next is how the four validities can be used to carefully evaluate the there different categories of claims.