11

Chapter 11: Quasi-Experimental Designs and Applied Research

Key Terms & Concepts

• Quasi-Experimental Design: Research designs that lack random assignment to treatment and control groups.

• Nonequivalent Control Group Design: Involves pre-existing groups where individuals are not randomly assigned.

• Regression to the Mean: Tendency of extreme scores to return closer to the average over time.

• Interrupted Time Series: Design assessing the effect of an intervention over multiple time points.

• Program Evaluation: Systematic method for assessing the design, implementation, and outcomes of a program.

• Needs Analysis: Process of identifying and evaluating needs before program implementation.

• Formative & Summative Evaluation: Formative evaluation occurs during program development while summative evaluation assesses effectiveness post-implementation.

• Cost-Effectiveness Analysis: Analysis to compare relative costs and outcomes of programs.

Beyond the Laboratory

Dual Functions of Applied Research

• Solves real-world problems.

• Increases basic knowledge and evaluates theory.

Research Example 33

• Independent Variable (IV): Color code of nutrition labels.

• Function #1: Improved non-dieters' use of color-coded labels: enhanced evaluation of health quality and food consumption.

• Function #2: Supported the importance of decision-making processes in health behaviors.

Applied Psychology in Historical Context

• Early Experimentalists: Faced pressures to demonstrate real-world relevance.

• Caffeine Study by Hollingworths:

  • Examined effects of caffeine in Coca-Cola.

  • Highlighted potential dangers: could mask need for sleep and be addictive.

  • Methodological notes: counterbalancing, double-blind, placebo controls.

  • Results: No adverse effects found on sleep quality, but caution advised regarding validity.

Design Problems in Applied Research

Ethical Dilemmas

• Issues of consent, privacy, and potential coercion.

Validity Challenges

• Internal Validity: May suffer due to ethical concerns or practical limitations.

• Between-Subjects Designs: Difficult to create equivalent groups.

• Within-Subjects Designs: Susceptible to order effects and participant attrition.

Quasi-Experimental Designs

• Lack causal conclusions.

• Offer less control and no random assignment.

• Types from previous chapters include:

  • Single-factor nonequivalent groups designs

  • Ex post facto factorial designs

  • P x E factorial designs.

  • All correlational research.

Nonequivalent Control Group Designs

• Typically include pretests and posttests.

  • Experimental: O1 T O2.

  • Nonequivalent Control: O1 O2.

• Random assignment is often not possible; groups may differ at baseline.

Regression to the Mean and Matching

• Pretest matching may affect outcomes:

  • Experimental group may score higher on pretest.

  • Control group lower than population mean.

  • Both groups might regress towards the mean, masking real treatment effects.

Research Examples with Nonequivalent Control Groups

Example 34

• IV1: Implementation of Play Streets.

• IV2: Before/after intervention perspective.

• DV: Amount of physical activity.

Example 35

• IV: Distance from San Francisco earthquake.

  • Experimental group: California.

  • Nonequivalent control: Arizona, no pretests.

• DV: Nightmare frequency; results suggest more nightmares in California but controlled for pre-existing differences.

Interrupted Time Series Designs

• Useful for evaluating the impact over time.

  • Basic structure: O1 O2 O3 O4 O5 T O6 O7 O8 O9 O10.

  • Best Outcome observed at post-intervention measurement.

Research Example 36

• Investigated effect of an incentive plan on productivity while ruling out alternative explanations.

Variations on Interrupted Time Series Designs

• Control Groups added: O1 O2 O3 O4 O5 T O6 O7 O8 O9 O10 O1 O2 O3 O4 O5 O6 O7 O8 O9 O10.

• Switching Replication: Introducing a second treatment at a different time demonstrates effects over more than one condition.

Adding Additional Measurements

• Introduce a second DV expected to remain unaffected by intervention to validate results.

Program Evaluation

• Reforms treated as experiments: Example from Connecticut speeding study.

• Comparison of effectiveness across control states; advocacy for experimental approaches.

Needs Analysis

• Involves planning programs based on:

  • Census data, resource surveys, potential user surveys, key informants, focus groups, community forums.

• Research Example 37: Assessment of healthy behaviors in the workplace focusing on employee data and program availability.

Monitoring Programs

Formative Evaluation

• Ongoing assessment during program implementation.

• Questions include: Was it implemented as planned? What is the program audit?

Summative Evaluation

• Focus on overall effectiveness post-implementation; can lead to quasi-experimental designs.

Cost-Effectiveness Analysis

• Evaluates costs relative to program effectiveness.

• Discusses differences between equally effective programs to determine the most cost-effective choice.

• Research Example 37 Continued: Evaluating wellness programs to establish cost-effectiveness, favoring those emphasizing support over expensive large-scale offerings.

Example from Prof. Ryan’s I/O Days

• Examined training programs with varying hours with respect to error rates.

• Analysis of training costs versus savings from error reduction leads to cost-benefit evaluations.

Summary

• Applied research helps clarify causes and find solutions to real-world challenges.

• Quasi-experimental designs, like nonequivalent control groups and interrupted time series, aid in studying situations where random assignment is impractical.

• Program evaluation methodologies critically assess the effectiveness of initiatives.