Research Designs

Program Evaluation

  • Definition: Research that assesses the effectiveness and efficiency of programs aimed at achieving positive impacts on a target population.

  • Purpose: Contributes to program improvement for initiatives such as:

    • New school curriculum

    • Anti-smoking campaigns

    • Healthy eating promotions

Key Components of Program Evaluation

  1. Needs Assessment

    • Identify problems in the target population.

  2. Program Theory Assessment

    • Determine underlying causes and how to address them.

  3. Process Evaluation

    • Evaluate the program's reach and implementation fidelity.

  4. Outcome Evaluation

    • Assess if intended outcomes are being achieved.

  5. Efficiency Assessment

    • Analyze the cost-effectiveness and potential improvements to the program.

Evaluation Standards

  • Utility: Evaluation should serve the information needs of intended users.

  • Feasibility: Evaluation should be realistic, prudent, and cost-effective.

  • Propriety: Evaluation should be conducted legally and ethically.

  • Accuracy: Evaluation should reveal and convey technically adequate information.

Research Design Types

True Experiments

  • Characteristics:

    • Involves random assignment to groups.

    • Allows inferences about causality.

    • Ideal for assessing independent and dependent variable relationships.

Quasi-Experiments

  • Definition: Designs that resemble experiments but lack random assignment or control groups.

  • Validity: Generally exhibit lower internal validity and should be used only when true experiments are not possible.

  • Examples of designs:

    • 1-group posttest-only

    • 1-group pretest-posttest

    • Non-equivalent control group

    • Interrupted time series

    • Control series

Internal Validity

  • Definition: The extent to which an independent variable causes changes in a dependent variable.

  • Threats to internal validity:

    • History effects: External events between measurements affect outcomes.

    • Maturation effects: Natural changes in the study population impact the results.

    • Testing effects: Effects of pretesting may alter posttest responses.

    • Instrumentation decay: Changes in measurement accuracy over time.

    • Regression toward the mean: Extreme scores that regress to the average on retesting.

Single Case Experimental Designs

  • Definition: Designed to evaluate the effects of an intervention on a single subject.

  • Types:

    • Reversal Design (ABA)

    • Measures behavior before, during, and after treatment to assess changes.

    • Multiple Baseline Design

    • Observes behaviors across different subjects, situations, or times without withdrawing treatment.

Developmental Research Designs

  • Purpose: Examine changes across the lifespan.

  • Types:

    1. Longitudinal Design

    • Same individuals studied over time; expensive and subject to high attrition.

    1. Cross-Sectional Design

    • Different age groups studied at one point in time; cheaper but risk cohort effects.

    1. Sequential Design

    • Combines longitudinal and cross-sectional methods; tests for cohort differences efficiently.

Types of Quasi-Experiments

Quasi-experiments are designs that resemble experiments but lack random assignment or control groups. They exhibit generally lower internal validity and should be used only when true experiments are not possible. Here are some examples of quasi-experimental designs:

  1. 1-group posttest-only

    • Involves a single group that is measured after an intervention has occurred.

  2. 1-group pretest-posttest

    • Measures the same single group before and after the intervention to assess change.

  3. Non-equivalent control group

    • Compares outcomes between a group that receives the intervention and a similar group that does not, but without random assignment.

  4. Interrupted time series

    • Involves multiple measurements taken before and after an intervention to track changes over time.

  5. Control series

    • Similar to the interrupted time series, but includes a control group to help account for external factors affecting the outcome.

Reversal Design (ABA)

  • Measures behavior before, during, and after treatment to assess changes. In this design, 'A' represents the baseline phase where the behavior is measured without intervention, and 'B' represents the intervention phase where the treatment is applied. After the intervention (B), the design may return to the baseline (A) to see if the behavior reverts to the original state, helping to demonstrate the effect of the intervention clearly.