OP

11

Chapter 11: Quasi-Experimental Designs and Applied Research

Key Terms & Concepts

  • Quasi-Experimental Design: Research designs that lack random assignment to treatment and control groups.

  • Nonequivalent Control Group Design: Involves pre-existing groups where individuals are not randomly assigned.

  • Regression to the Mean: Tendency of extreme scores to return closer to the average over time.

  • Interrupted Time Series: Design assessing the effect of an intervention over multiple time points.

  • Program Evaluation: Systematic method for assessing the design, implementation, and outcomes of a program.

  • Needs Analysis: Process of identifying and evaluating needs before program implementation.

  • Formative & Summative Evaluation: Formative evaluation occurs during program development while summative evaluation assesses effectiveness post-implementation.

  • Cost-Effectiveness Analysis: Analysis to compare relative costs and outcomes of programs.

Beyond the Laboratory

Dual Functions of Applied Research

  • Solves real-world problems.

  • Increases basic knowledge and evaluates theory.

Research Example 33

  • Independent Variable (IV): Color code of nutrition labels.

  • Function #1: Improved non-dieters' use of color-coded labels: enhanced evaluation of health quality and food consumption.

  • Function #2: Supported the importance of decision-making processes in health behaviors.

Applied Psychology in Historical Context

  • Early Experimentalists: Faced pressures to demonstrate real-world relevance.

  • Caffeine Study by Hollingworths:

    • Examined effects of caffeine in Coca-Cola.

    • Highlighted potential dangers: could mask need for sleep and be addictive.

    • Methodological notes: counterbalancing, double-blind, placebo controls.

    • Results: No adverse effects found on sleep quality, but caution advised regarding validity.

Design Problems in Applied Research

Ethical Dilemmas

  • Issues of consent, privacy, and potential coercion.

Validity Challenges

  • Internal Validity: May suffer due to ethical concerns or practical limitations.

  • Between-Subjects Designs: Difficult to create equivalent groups.

  • Within-Subjects Designs: Susceptible to order effects and participant attrition.

Quasi-Experimental Designs

  • Lack causal conclusions.

  • Offer less control and no random assignment.

  • Types from previous chapters include:

    • Single-factor nonequivalent groups designs

    • Ex post facto factorial designs

    • P x E factorial designs.

    • All correlational research.

Nonequivalent Control Group Designs

  • Typically include pretests and posttests.

    • Experimental: O1 T O2.

    • Nonequivalent Control: O1 O2.

  • Random assignment is often not possible; groups may differ at baseline.

Regression to the Mean and Matching

  • Pretest matching may affect outcomes:

    • Experimental group may score higher on pretest.

    • Control group lower than population mean.

    • Both groups might regress towards the mean, masking real treatment effects.

Research Examples with Nonequivalent Control Groups

Example 34

  • IV1: Implementation of Play Streets.

  • IV2: Before/after intervention perspective.

  • DV: Amount of physical activity.

Example 35

  • IV: Distance from San Francisco earthquake.

    • Experimental group: California.

    • Nonequivalent control: Arizona, no pretests.

  • DV: Nightmare frequency; results suggest more nightmares in California but controlled for pre-existing differences.

Interrupted Time Series Designs

  • Useful for evaluating the impact over time.

    • Basic structure: O1 O2 O3 O4 O5 T O6 O7 O8 O9 O10.

    • Best Outcome observed at post-intervention measurement.

Research Example 36

  • Investigated effect of an incentive plan on productivity while ruling out alternative explanations.

Variations on Interrupted Time Series Designs

  • Control Groups added: O1 O2 O3 O4 O5 T O6 O7 O8 O9 O10 O1 O2 O3 O4 O5 O6 O7 O8 O9 O10.

  • Switching Replication: Introducing a second treatment at a different time demonstrates effects over more than one condition.

Adding Additional Measurements

  • Introduce a second DV expected to remain unaffected by intervention to validate results.

Program Evaluation

  • Reforms treated as experiments: Example from Connecticut speeding study.

  • Comparison of effectiveness across control states; advocacy for experimental approaches.

Needs Analysis

  • Involves planning programs based on:

    • Census data, resource surveys, potential user surveys, key informants, focus groups, community forums.

  • Research Example 37: Assessment of healthy behaviors in the workplace focusing on employee data and program availability.

Monitoring Programs

Formative Evaluation

  • Ongoing assessment during program implementation.

  • Questions include: Was it implemented as planned? What is the program audit?

Summative Evaluation

  • Focus on overall effectiveness post-implementation; can lead to quasi-experimental designs.

Cost-Effectiveness Analysis

  • Evaluates costs relative to program effectiveness.

  • Discusses differences between equally effective programs to determine the most cost-effective choice.

  • Research Example 37 Continued: Evaluating wellness programs to establish cost-effectiveness, favoring those emphasizing support over expensive large-scale offerings.

Example from Prof. Ryan’s I/O Days

  • Examined training programs with varying hours with respect to error rates.

  • Analysis of training costs versus savings from error reduction leads to cost-benefit evaluations.

Summary

  • Applied research helps clarify causes and find solutions to real-world challenges.

  • Quasi-experimental designs, like nonequivalent control groups and interrupted time series, aid in studying situations where random assignment is impractical.

  • Program evaluation methodologies critically assess the effectiveness of initiatives.