Experimental Design in Applied Behavior Analysis
Briefing Document: Experimental Design in Applied Behavior Analysis
Executive Summary
Provides a synthesis of the principles, methodologies, and ethical considerations of experimental design within Applied Behavior Analysis (ABA).
Central Objective: To demonstrate a functional relation, proving that a specific intervention (independent variable) is directly responsible for a change in a target behavior (dependent variable).
Predominant Methodology: Single-Case Experimental Design (SCED), where each participant serves as their own control, relying on baseline logic—comprising prediction, verification, and replication—to establish experimental control.
High Internal Validity: Achieved by systematic control of extraneous and confounding variables to reveal true effects of the intervention.
Four Primary SCEDs:
Reversal Design: Systematically introduces and withdraws an intervention to show a functional relation.
Multiple Baseline Design: Implements an intervention across behaviors, settings, or subjects in a staggered manner, optimal for irreversible behavior changes.
Multielement Design: Rapidly alternates between two or more interventions to compare their effectiveness.
Changing Criterion Design: Applies an intervention in a stepwise fashion to an existing behavior.
Specialized Analytic Methods: Component, comparative, and parametric analyses for nuanced investigations into treatment packages and intervention dosages.
Ethical Framework: Governed by the BACB Code Standard 6, focusing on client welfare, informed consent, and data integrity.
1. Foundational Concepts in Experimental Design
Dependent vs. Independent Variables
Dependent Variable (DV):
The target behavior of interest measured in an experiment; considered dependent on the independent variable's manipulation.
Examples: heart rate, rate of response, aggressive behaviors.
Independent Variable (IV):
The manipulated aspect of the environment to assess its effect on the DV; also known as treatment or intervention.
Examples: specific procedures like Differential Reinforcement of Other behaviors (DRO), shaping, token economy, extinction.
Internal vs. External Validity
Internal Validity:
The extent to which an experiment demonstrates that changes in the DV arise directly from the manipulation of the IV without influence from uncontrolled variables.
Synonymous with experimental control; prioritized in ABA research.
External Validity:
The extent to which study results generalize to other subjects, settings, or behaviors. Confirmed through replication.
Types of Scientific Replication
Direct Replication: Exact replication of a previous study.
Intrasubject Direct Replication: Uses the same participants, enhancing reliability.
Intersubject Direct Replication: Uses different, relevant participants demonstrating generality.
Systematic Replication: Intentionally alters aspects of prior experiments (e.g., participant demographics or IV) to verify reliability and external validity.
2. Threats to Internal Validity
Uncontrolled Variables
Extraneous Variables: Aspects of the environment that, if uncontrolled, may cause variation in results (e.g., lighting, noise).
Confounding Variables: Uncontrolled factors that exert influence on the DV, such as participant's lack of sleep or environmental disruptions (e.g., earthquakes).
Categories of Threats to Internal Validity
Measurement Confounds: Issues in measuring outcomes.
Examples: Observer drift, reactivity, observer bias/expectations.
Mitigation Strategies:
Increase observer training.
Use permanent product measures.
Implement double-blind controls, unobtrusive measures, and continuous data collection.
Independent Variable (IV) Confounds: Issues muddling IV effects, typically in complex treatment packages.
Mitigation Strategies:
Utilize simple treatment packages.
Ensure clear operational definitions and high treatment integrity.
Subject Confounds: Participant-related variables affecting results.
Examples: Maturation, history, attrition, practice effects.
Setting Confounds: Uncontrolled variables inherent in treatment settings.
Mitigation Strategies:
Maintain consistency in all potential sources of reinforcement.
Prevent bootleg reinforcement (unearned access to reinforcement).
3. The Logic and Features of Single-Case Experimental Designs (SCEDs)
SCEDs: Each participant serves as their own control; IV effects are compared against baseline data. Most ABA experiments utilize SCEDs.
Baseline Logic
A three-part reasoning process in SCEDs:
Prediction: Projecting a stable data path (level and trend) will persist without intervention.
Verification: Showing that the DV returns to baseline levels upon IV removal.
Replication: Reinforcing the functional relation by reintroducing the IV.
Affirmation of the Consequent: Initial response change upon IV introduction indicates a functional relationship.
Baseline Data Patterns
Descending Baseline:
If increasing behavior, implement IV immediately.
If decreasing behavior, delay IV implementation.
Ascending Baseline:
If increasing behavior, wait for IV.
If decreasing behavior, implement IV immediately.
Variable Baseline: Non-stable data; control variability before IV.
Stable Baseline (Steady State Responding): Ideal for IV introduction due to consistent trends.
4. Core Single-Case Experimental Designs
4.1 Reversal Design (A-B-A Design)
Description: Systematically introduces and withdraws IV across at least three phases: Initial Baseline (A), Intervention (B), Return to Baseline (A).
Functional Relation: Demonstrated by changes from baseline to treatment and back to baseline.
Limitations:
Irreversibility: Not applicable for unlearnable behaviors (e.g., reading).
Ethical Concerns: Inappropriate to withdraw effective interventions for dangerous behaviors.
Variations of Reversal Design:
Repeated Reversals (A-B-A-B): Strengthen evidence with multiple phase presentations.
B-A-B Reversal: Starts with treatment for urgent cases.
Multiple Treatment Reversal (A-B-C): Compare multiple IVs to baseline.
NCR Reversal Technique: Uses Noncontingent Reinforcement.
DRO/DRA/DRI Reversal Technique: Employ non-contingent reinforcement types.
4.2 Multielement Design (Alternating Treatments Design)
Description: Rapidly alternates between two or more IVs.
Functional Relation: Shown when one data path consistently trends higher or lower than others.
Use Cases: Ideal for comparing treatment efficacy or conducting functional analyses.
Advantages: Requires no treatment withdrawal or baseline stability.
Disadvantages: Risks interference from multiple treatments and unnatural rapid alternation.
4.3 Multiple Baseline Design (MBL)
Description: Evaluates one IV across different behaviors, settings, or subjects in staggered implementations.
Functional Relation: Change in DV occurs only after the IV is introduced to that specific baseline.
Use Cases: Appropriate for irreversible behaviors or when withdrawal is unethical.
Subtypes:
Across Behaviors, Across Settings, Across Subjects.
Weaker variations: Multiple Probe, Delayed, Nonconcurrent (least strong).
4.4 Changing Criterion Design
Description: An initial baseline followed by stepwise treatment phases with successive criterion changes.
Functional Relation: Demonstrated by conformity of DV to new criteria.
Use Cases: Effective for shaping within an existing response topography.
Guidelines: Vary phase lengths and criterion change magnitudes.
5. Advanced Analyses and Design Comparisons
Single-Case Designs vs. Group Designs
Group Designs:
Participants assigned to experimental and control groups; useful for large evaluations.
Masks individual performance and variability.
Single-Case Designs:
Used by behavior analysts for clinically valid changes.
Allows individual performance representation, intrasubject replication, and flexibility.
Specialized Analytic Methods
Comparative Analysis: Compares two or more IVs for effectiveness (e.g., multielement design).
Component Analysis: Assesses individual components of a multi-component treatment package.
Drop-Out Analysis: Systematically removes components.
Add-In Analysis: Tests components individually before full package presentation.
Parametric Analysis: Manipulates IV dosages to compare effects.
Nonparametric Analysis: Compares presence vs. absence of the IV.
6. Framework for Evaluating and Interpreting Research
Key Questions for SCED Evaluation
Is there a reliable functional relation between the IV and DV?
Are the data accurate measures of target behavior?
Is the graph appropriately scaled?
Is the target behavior stable within phases?
Were baseline conditions appropriate?
Was an appropriate experimental design used?
Did the study control for confounding variables?
Is the study relevant, and are outcomes meaningful?
Possible Errors:
Type I Error (False Positive): Concluding IV impacted DV when it did not.
Type II Error (False Negative): Concluding IV did not impact DV when it did indeed.
Is there strong Interobserver Agreement (IOA)?
Is the IV socially valid?
Has the participant's life improved?
Have improvements been generalized and maintained?
Was maintenance and generalization reported?
How does the study compare to similar research (external validity)?
Does the study contribute to the advancement of the field?
7. Ethical Considerations in Research (Code Standard 6)
Ethical Compliance
Compliance and Review (6.01, 6.02):
Adherence to relevant laws; approval from a formal review committee required before research.
Participant Rights and Welfare (6.03, 6.04, 6.05):
Prioritize client welfare; informed consent must be obtained, communicating purpose, procedures, risks, benefits, and confidentiality limits.
Assent from those unable to provide consent; confidentiality of participant data must be maintained.
Researcher Integrity and Competence (6.06, 6.07, 6.08, 6.09):
Conduct research only after sufficient training; disclose and address conflicts of interest; acknowledgment of all contributors; prohibition of plagiarism.
Data Management and Accuracy (6.10, 6.11):
Comply with standards for data handling; prohibit fabricating or falsifying data; correct errors appropriately, presenting data wholly where possible.
Concept Map of Single-Case Experimental Designs (SCED)
Central Concept
SCED: Experimental methodology where individuals serve as their own controls, and intervention effects are compared to individual baseline data.
Branch 1: Core Components & Validity
Variables:
IV: Intervention manipulated by researcher.
DV: Measured target behavior assessing IV effects.
Internal Validity (Priority):
Definition: Changes in DV directly result from IV, unaffected by uncontrolled variables.
Threats: Measurement (observer drift, reactivity), IV (complex packages), Subject (maturation, history), Setting (bootleg reinforcement).
External Validity:
Definition: Generalizability to other subjects/settings/behaviors.
Established via Replication: Direct (intra/inter-subject) and systematic.
Branch 2: Baseline Logic (Methodological Reasoning)
Steady State Strategy: Repeatedly expose DV to IV until stable responding observed.
Three Elements of Baseline Logic:
Prediction: Data will remain unchanged if current conditions persist.
Verification: Asserts IV responsibility by returning behavior to baseline.
Replication: Treatment effect reproducibility ensures reliability.
Affirmation of the Consequent: Behavior changes following IV introduction suggests control.
Baseline Data Patterns:
Describes circumstances for initiating intervention based on trends.
Branch 3: The Four Main Experimental Designs
Reversal Design (A-B-A):
Structure: Systematic IV introduction and withdrawal.
Strength: Demonstration of functional relations.
Limitations & Variations: Irreversible behaviors, ethical concerns, repeats, and control conditions.
Multielement Design (Alternating Treatments):
Structure: Rapid alternation of IVs.
Strength: No required treatment withdrawal, adaptability to unstable data.
Limitations: Treatment interference risks.
Multiple Baseline Design (MBL):
Structure: Staggered IV implementation across tiers.
Strength: Ethical in irreversible cases.
Changing Criterion Design:
Structure: Stepwise treatment criterion shifts.
Key Requirement: Existing behavior repertoire.
Branch 4: Analytic Approaches
Comparative Analysis: Compare distinct IVs for effectiveness.
Component Analysis: Isolate components affecting behavior change.
Drop-out/add-in strategies.
Parametric Analysis: Assess varying dosages of a single IV.
Nonparametric Analysis: Compare IV presence vs. absence.
Branch 5: Ethical Compliance (Code Standard 6)
Research must be approved; informed consent and confidentiality prioritized.
Research integrity must be maintained, including accurate data management and prohibition of unethical practices.
Analogy to Solidify Understanding
The Relationship of Baseline Logic to a Scientific Trial:
Prediction: Prosecutor claims no change without action.
Affirmation: Evidence of IV effect post-intervention.
Verification: Remove IV to prove behavior returns to baseline.
Replication: Repeat to reinforce intervention's reliability.
Study Guide: Single-Case Experimental Designs and Validity Quiz
Key Concepts in Experimental Design
Difference between IV and DV:
DV is the measured target behavior; IV is the manipulated environmental factor impacting the DV.
Internal vs. External Validity:
Internal validity reflects direct IV impacts on DV; external validity assesses generalizability. Priority is internal validity.
Subject confounds threatening internal validity:
Maturation (natural participant changes) and history (environmental shifts).
Core components of baseline logic:
Prediction, verification, replication.
Stable baseline: Optimal for IV introduction due to clear observation potential.
When to avoid reversal designs:
For irreversible or dangerous behaviors.
Functional relation in multiple baseline design:
Change occurs exclusively when IV is introduced at staggered baselines.
Purpose of multielement design:
Rapidly compare IV efficacy; no overlap in data paths indicates effectiveness.
Difference between component and parametric analysis:
Component isolates parts of a treatment; parametric manipulates IV dosage.
Distinction between Type I and Type II error:
Type I (False Positive): Concludes IV affected DV when it didn't. Type II (False Negative): Concludes no effect when there was one.
Answer Key
The dependent variable (DV), or target behavior, is what is measured to see if it changes due to the independent variable's manipulation. The independent variable (IV), also known as the treatment or intervention, refers to environmental aspects manipulated to assess their effects on the dependent variable.
Internal validity refers to the degree changes in the DV are attributed to the IV, establishing experimental control. External validity gauges the applicability of study results to other contexts. Prioritization lies with internal validity.
Maturation indicates natural growth-related changes, and history pertains to shifts in the participant's surroundings affecting the targeted behaviors during the study.
The three core components in baseline logic are prediction (data stability), verification (iv's effectiveness return to baseline upon removal), and replication (reproduce treatment results).
A stable baseline indicates no trends, representing the clearest observation point to monitor IV influence on the DV.
Reversal designs are not suitable for irreversible behaviors (e.g., learning to read) or critical dangerous behaviors, where ethical implications make treatment removal inappropriate.
In a multiple baseline design, functional relations are shown as changes occur strictly when the IV is applied to a specific baseline situation while other baselines remain unchanged.
Multielement designs swiftly allow for comparative efficacy analysis of multiple IVs; functional relations are clear through separated data comparisons.
Component analysis identifies actionable parts of a treatment package, whereas parametric manipulations change the dosage levels of a single IV to assess its impact.
Type I error (false positive) suggests that IV impacted DV when it hadn't, while Type II error (false negative) suggests it did not when it actually did.