SI- Threats to Internal and External Validity

I. Understanding Validity

• Internal Validity: Ensures that observed changes in the dependent variable result only from the manipulation of the independent variable.

• External Validity: Ensures that research findings can be generalized to other settings, populations, or times.

II. Threats to Internal Validity

1. History

   • Occurs when uncontrolled events influence outcomes between pretest and posttest.

   • Example: A natural disaster occurring during a stress-reduction intervention.

2. Maturation

   • Refers to changes in subjects over time unrelated to the study (e.g., aging, fatigue).

   • Particularly problematic in long-term studies.

3. Attrition (Mortality)

   • Occurs when participants drop out of the study, potentially skewing results.

   • Solution: Plan for sufficient sample size to account for dropouts.

4. Testing

   • Repeated testing can influence outcomes as subjects learn from previous assessments.

   • Example: Improved posttest scores due to familiarization rather than treatment.

5. Instrumentation

   • Changes in measurement tools or data collection techniques can affect outcomes.

   • Example: Malfunctioning equipment or altered survey wording.

6. Statistical Regression

   • Occurs when subjects selected for extreme scores naturally move closer to the mean upon retesting.

   • Solution: Avoid selecting participants solely based on extreme scores.

7. Selection Bias

   • Results from non-random assignment, causing systematic differences between groups.

   • Solution: Use random assignment to minimize this risk.

8. Placebo Effect

   • When subjects expect improvement, they may report better outcomes even if the treatment is ineffective.

   • Solution: Use control groups and blinding techniques.

III. Threats to External Validity

1. Sample to Population Effect

   • Occurs when the sample does not adequately represent the target population.

   • Solution: Ensure diverse and representative sampling.

2. Description of Experimental Treatment

   • Lack of detailed explanation may hinder replication and limit generalizability.

   • Solution: Clearly describe treatment protocols.

3. Exposure to Multiple Treatments

   • Participants may encounter multiple variables, making it hard to isolate individual effects.

4. Hawthorne Effect

   • Subjects alter behavior simply because they know they are being studied.

   • Solution: Include control groups and limit subject awareness of observation.

5. Researcher Effect

   • The researcher's behavior or characteristics may influence participants.

   • Solution: Standardize procedures and use multiple researchers when possible.

6. Pretest Effects

   • The act of taking a pretest may influence participants’ responses on subsequent assessments.

   • Solution: Use alternate forms or minimize reliance on pretesting.

IV. Key Takeaways

• Internal validity ensures that study outcomes are due to the independent variable and not external factors.

• External validity ensures findings apply beyond the study conditions.

• Employ strategies such as randomization, clear protocols, and control groups to reduce threats to validity.