Experimental Methods

Control Techniques in Experimental Research

  • Importance of Control:

    • Ideal scenario involves random selection and random assignment of participants.

    • However, random selection is rarely practical. Emphasis is placed on establishing cause-effect relationships rather than generalizability.

    • Common sampling methods:

    • Purposive sampling

    • Convenience sampling

Extraneous Variables

  • Definition: Extraneous variables are those that compete with the independent variable in explaining outcomes. They need to be controlled to maintain the integrity of the experiment.

  • Objective: Ensure that any change in the dependent variable is attributable solely to the independent variable by holding extraneous variables constant across groups.

Control Techniques at the Beginning of an Experiment

  • Random Assignment

    • Definition: Participants have an equal chance of being assigned to any group, equating groups on known and unknown variables.

    • Significance: Adjusts for systematic group differences, but does not eliminate extraneous variables, rather distributes them evenly.

    • Example: Comparison between non-random and random assignment indicates reduced systematic differences (e.g., IQ scores).

    • Feasibility: Sometimes impractical; therefore, matching techniques may be employed.

  • Matching

    • Definition: Participants are equated based on certain characteristics (e.g., age, sex, or other demographics).

    • Limitation: Participants may differ on unmeasured variables. Combining matching with random assignment improves reliability.

Matching Techniques

  • Holding an Extraneous Variable Constant

    • All participants have the same level of an extraneous variable (e.g., controlling for sex).

    • Disadvantages: Restricts participant pool and generalizability.

  • Blocking

    • Definition: Incorporating an extraneous variable into the research design.

    • Example: Group by age when studying language acquisition effects, recognizing different sensitivities at varying ages.

  • Yoked Control

    • Definition: Matches participants based on the timing of events or treatments.

    • Example: Participants paired to ensure simultaneous exposure to events (e.g., social media breaks).

  • Individual Matching

    • Definition: Participants are matched on a case-by-case basis before random assignment to groups.

    • Limitations:

    • Difficulty in selecting match variables.

    • Increasing difficulty with more matching variables leading to decreased generalizability.

Control Techniques During Experiments

  • Three main methods:

    • Counterbalancing

    • Controlling for Participant Effects

    • Controlling for Experimenter Effects

Counterbalancing

  • Important for Within Participants Designs.

  • Rationale: Administer treatments in varied sequences to prevent sequencing effects (e.g., practice might enhance performance in later conditions).

Types of Sequencing Effects

  • Order Effects: The order of administration impacts outcomes.

    • Example: Performance increase due to prior exposure.

  • Carry-Over Effects: Prior conditions influence subsequent treatments.

    • Example: Relaxation from one treatment affecting another's outcomes.

Types of Counterbalancing

  • Randomized Counterbalancing:

    • All participants experience varied sequences, randomized to mitigate sequencing effects.

  • Intra-subject Counterbalancing:

    • Participants experience treatment conditions in both original and reversed orders.

  • Complete Counterbalancing:

    • Every possible order of treatment is utilized, with equal participant distribution.

  • Incomplete Counterbalancing:

    • Some but not all possible sequences are utilized; most common in practice.

Controlling for Participant Effects

  • Double-Blind Placebo Method:

    • Neither the experimenter nor participant knows treatment conditions, eliminating biases.

  • Techniques for Insight into Participant Perceptions:

    • Retrospective Verbal Reports:

    • Post-experiment interviews to understand participants' thoughts on treatment expectations and outcomes.

    • Concurrent Verbal Reports:

    • Real-time reporting of thoughts during the experiment.

    • Types include:

      • Sacrifice Groups: Participants intermittently interviewed.

      • Concurrent Probing: Insights gathered after each trial.

      • Think-Aloud Technique: Participants verbalize thoughts, though experts may not access automatic processes easily.

Controlling for Experimenter Effects

  • Experimental Effects: Unintentional biases caused by the experimenter.

  • Techniques include:

    • Control for recording errors via training or technology.

    • Maintain one experimenter across conditions.

  • Blind Technique:

    • The experimenter knows hypotheses but is blind to participant conditions.

  • Partial Blind Technique and Automation:

    • Reduce participant interaction to minimize bias.

Importance of Control Techniques

  • Validity and Reliability: Research designs must effectively manage extraneous variables for credible findings.

  • Replication strengthens research reliability by revealing effects of extraneous variables over multiple trials.

Research Designs Overview

  • Research design refers to the strategy for investigating a research problem, specifying data collection and analytical approaches.

  • Strong Designs Criteria:

    • Pre-tests

    • Control Groups

    • Random Assignment

Weak Experimental Designs

  • Weak designs control few extraneous variables, thus provide limited cause and effect evidence.

  • One-Group Post-test Only Design:

    • One group post-test after treatment, significant limitations due to lack of control group or pre-test.

  • One-Group Pretest-Posttest Design:

    • Improvement over the previous, but still weak as rival hypotheses exist.

  • Posttest-Only with Non-Equivalent Groups:

    • Comparison of outcomes in a treatment group and an uncontrolled group raises potential selective bias.

Strong Experimental Designs

  • Effectively manage extraneous variables to provide sound cause and effect evidence.

  • Components include:

    • Protests for measuring dependent variables pre-treatment.

    • Compare treatment and control groups for validity of interventions.

    • Randomly assign to avoid systematic differences.

Types of Strong Experimental Designs

  • Between-Participants Design:

    • Different groups for different treatments.

  • Within-Participants Design:

    • Same participants across all conditions; improves control of individual differences and is managed by counterbalancing.

  • Mixed Designs:

    • Incorporate aspects of both between and within participants designs; often involve pre-test and post-test measures.

Factorial Designs

  • Definition: Factorial designs involve studying two or more independent variables (IVs) to assess their separate and joint effects on a dependent variable (DV).

  • Design Variants:

    • Only Between-Participants Variables: Subjects are exposed to only one level of each IV.

    • Only Within-Participants Variables: All subjects are exposed to all levels of each IV.

    • Mixed Factorial Designs: Combines both between and within-participants variables.

Types of Effects in Factorial Designs

  • Main Effect:

    • Represents the influence of one independent variable on the dependent variable while ignoring the second IV.

    • There exists one main effect for each IV in the study.

  • Interaction Effect:

    • Denotes the joint or interactive influence of two or more IVs on the DV.

    • Graphically represented by non-parallel lines; indicating that the effect of one IV depends on the level of another IV.

Importance of Experimental Design Selection

  • Decision factors include:

    • Ability to Test Multiple Variables: The interaction between variables can yield richer hypotheses.

    • Control Over Extraneous Variables: Including extraneous variables as IVs enhances internal validity.

    • Complexity and Feasibility: More than two IVs may complicate the study design and logistics, making interpretation difficult.

  • Statistical Power:

    • The probability of correctly rejecting the null hypothesis when it is false.

    • Influence Factors: sample size, alpha level (often set at p < 0.05).

Sample Size Considerations

  • Significance of Sample Size:

    • Larger sample sizes facilitate the detection of effects that exist within the population.

    • Sample size can be calculated using reference tables or software like G*Power.

    • Power analysis is necessary for determining the required sample size.

Hypotheses in Research

  • Null Hypothesis: Assumes no relationship among the variables; tested in experiments.

  • Alternate Hypothesis: Posits there is a relationship between IV and DV; accepted when null is rejected after statistical testing.

Research Procedures

Detailed Steps of Research Procedure

  1. Planning and Scheduling: Participants should be informed about when and how the study will occur, including potential risks and benefits.

  2. Informed Consent: Participants receive a consent information statement detailing the study's purpose, procedures, risks, and benefits.

  3. Data Collection: The experiment should follow the designed procedure accurately to assure reproducibility.

Debriefing

  • Required post-experiment interaction to inform participants about the study’s purpose and any deception involved. This can also help assess treatment effectiveness.

Pilot Studies

  • Definition: A brief trial run of the experiment with a small group of subjects before the actual study.

  • Benefits of Pilot Studies:

    • Determining procedure feasibility and refining instructions.

    • Ensuring the manipulation of IVs is valid.

    • Provides an opportunity for preliminary data collection for statistical analysis.

    • Despite increasing the chance of successful main studies, pilot studies do not guarantee success.

Institutional Approval and Participant Selection

Institutional Approval

  • Obtaining necessary approvals from ethical review boards is essential before conducting experiments with human or animal subjects.

Participant Recruitment

  • Sample selection impacts validity and generalizability of results. Parameters to consider include:

    • Defining inclusion and exclusion criteria based on the population of interest.

    • Ethical requirements for special populations (e.g., minors).

Sample Size Consideration

  • Balancing the need for sufficient participants to obtain statistically significant results with practical constraints like costs and logistics.

Experimental Procedure Execution

  • Essential to adhere closely to planned procedures for accurate, replicable results. Deviations should be documented for transparency.

Equipment and Tools

  • Selection of appropriate apparatus depends on the kind of manipulation and measurement required for the IV and DV, respectively (e.g., simulators, neuroimaging equipment).

Quasi-Experimental Design

  • Definition: A research design that applies experimental procedures but does not control all extraneous variables. Typically lacks random assignment.

  • Causal inference: Provided by ruling out rival hypotheses via different methods:

    • Identification and study of plausible threats to internal validity.

    • Control through design (e.g., adding pre-tests, additional control groups).

    • Coherent pattern matching (making predictions that few rival hypotheses can explain).

Sub-Types of Quasi-Experiments

  1. Non-equivalent comparison group design

    • Most common quasi-experimental design.

    • Involves experimental and control groups without random assignment.

    • Use of pre-test is crucial to determine group equivalence; significant pre-test differences may indicate selection bias.

    • Matching and statistical control techniques are critical in these studies.

  2. Time-series design

    • Assess treatment effect by comparing pre- and post-test scores of a single group.

    • Researcher analyses patterns for discontinuity in dependent measures.

  3. Regression-discontinuity design

    • Assigns participants to groups based on assignment variable scores.

    • Assesses treatment effect by examining discontinuity in regression lines.

Internal Validity Threats in Quasi-Experimental Designs

  • Importance of ruling out rival hypotheses:

    • Maintains internal validity by identifying plausible threats.

    • Employ control designs to strengthen findings: pre-tests and multiple groups can be crucial.

    • Coherent pattern matching allows complex predictions reducing alternative explanations.

Assessment of Treatment Effects

  • Method of evaluating involves changes in the main level and/or slope of pre-test and post-test responses.

  • Signs of treatment effect include:

    • A discontinuous line in data representation indicating interrupted trends.

    • Importance of multiple measures in establishing genuine changes.

Single-Case Designs

  • Definition: Research designs that involve one participant or a single group as a unit in the experiment.

  • Comparison is solely within the participant selves, using time series designs.

  • Collect multiple data points before and after treatment is introduced.

  • Assesses treatment effect based on the assumption that without treatment, pre-treatment patterns continue (the counterfactual).

Different Types of Single-Case Designs

  1. A design

    • Simplest form: response to treatment compared to baseline.

    • Ethical issues arise when ending on a baseline without offering treatment.

  2. ABAB Design

    • An extension of A, the treatment effect is reintroduced at the end.

    • Adds to ethical soundness by ensuring participants have continued access to treatment.

  3. Experimental and Therapeutic Criteria

    • Experimental Criterion: Requires repeated demonstration of behavioral change when treatment is present.

    • Therapeutic Criterion: Focuses on clinical significance of intervention for individual clients.

    • Emphasis on shifting clinical status from disorder to normal functioning.

Historical Context of Single-Case Research

  • Key figures in the development of single case designs:

    • Ebbinghaus: Memory research using himself as a participant.

    • Pavlov: Classical conditioning experiments with dogs to illustrate learning.

    • Piaget: Observations of children’s cognitive development.

    • Skinner: Advocated for intensive study of individuals to derive general principles (quote: "It is better to study an individual for 1000 hours than 1000 individuals for an hour each").

Notable Aspects of Single-Case Studies

  • Distinguishing between single-case designs and case studies:

    • Single-case designs evaluate treatment effects.

    • Case studies provide in-depth descriptive analysis without assessing treatment.

Advanced Single-Case Design Strategies

Time Series Design

  • Collection of Data Points: Focuses on responses collected before and after treatment interventions.

  • Assumption: Pre-treatment patterns would maintain without additional influences.

Counterfactual Concept

  • The hypothetical situation predicting behavior without treatment is termed the counterfactual.

API and AB Design

  • A and B Conditions: A encompasses baseline behavior measures; B represents measures post-treatment. Reintroducing A allows observation of potential behavior reversion to pre-treatment levels, indicating treatment effectiveness.

Limitations and Ethical Considerations

  • Ending a study on a baseline may lead to ethical dilemmas (e.g., withdrawing treatment).

  • Multiple Baseline Design: Alternative design in which treatments are given to several targets (participants, settings, or behaviors) to observe staggered treatment effects.

Other Design Extensions

  • Interaction Designs: Test combined effects of multiple treatments (e.g., rewards vs. punishments).

  • Changing-Criterion Design: Gradually increases performance criteria to shape participant behavior through successive treatment periods.

  • Multiple-Baseline Design: Effectively administered across several conditions, helping to establish treatment effects against history threats.