Notes on Experimental Designs and Internal Validity

Research Goals and Claims

• Describing behavior: Making claims about frequency (e.g., prevalence of depression).
• Predicting behavior: Making claims about associations (e.g., is depression related to spending time alone?).
• Explaining behavior: Making causal claims (e.g., does social isolation cause depression?).

Evaluating Causal Claims

1. External validity: Generalizability of findings.
2. Construct validity: Accuracy of the operational definitions used in the study.
3. Statistical validity: Appropriateness of the statistical analyses used.
4. Internal validity: Assurance that the independent variable truly caused the difference observed in the dependent variable.

Internal Validity

• Key Question: Did the independent variable (IV) actually cause the observed change in the dependent variable (DV)?
• Example: Does gratitude decrease materialism levels?
• IV: Gratitude (manipulated variable).
• DV: Materialism (measured variable).

Threats to Internal Validity

1. Confounds: Variables that vary systematically with the levels of the IV.
2. Selection effects: Differences in participant characteristics across conditions, particularly problematic when participants self-select their conditions.
3. Order effects: Impact of the order in which participants experience conditions.
4. Researcher bias: Researchers’ expectations alter their interpretation of study results.
5. Demand characteristics: Participants alter their behavior based on their guesses about the purpose of the study.
6. Placebo effects: Improvement due to the belief in receiving effective treatment.

Solutions to Internal Validity Threats

• Confounds & Selection Effects: Use of random assignment or matched groups design.
• Order Effects: Implementing counterbalancing to manage the effects of the order in which conditions are experienced.
• Researcher Bias & Demand Characteristics: Conducting double-blind studies to minimize bias from both participants and researchers.
• Placebo Effects: Incorporating a placebo control group into the experiment.

Maturation Threat

• Changes in participants naturally over time can confound results.
• Solution: Use a comparison group to help rule out changes due to maturation.

History Threat

• External events affecting all participants during the study timeline can confound results.
• Solution: Again, use a comparison group for better control.

Regression Threat

• The tendency for extreme scores to return to their average over time can skew results.
• Solution: Include control groups to distinguish between true effects and regression.

Attrition Threat

• Systematic drop-out of participants can skew results, particularly if individuals with specific characteristics are more likely to leave.
• Dealing with this threat: Check pre-test scores of both drop-outs and those who remained, particularly focusing on extreme scores.

Testing Threat

• Changes in scores might occur simply due to repeated testing rather than actual changes in the construct measured.
• Solution: Use posttest-only designs or different measures at pretest and posttest.

Instrumentation Threat

• Changes in how the DV is measured can lead to apparent changes in the score that are not due to the treatment.
• Solution: Ensure consistent measurement practices and validations across time.

Summary of Internal Validity Threats

• The internal validity of study designs can be compromised by confounds, selection effects, order effects, researcher bias, demand characteristics, placebo effects, maturation threats, history threats, regression threats, attrition threats, testing threats, and instrumentation threats.

Practice Questions

• Participate in discussions or tests to evaluate understanding of the concepts mentioned above by utilizing interactive platforms (e.g., wooclap.com).