Chapter 5 - How to Evaluate Interventions in Applied Settings

Assessment alone doesn’t reveal causes

• assessment alone is valuable in identifying if a change occurred

  • doesn’t tell us what caused the change

• why is knowing causes important?

  • it extends our knowledge base

  • benefits society

• evaluating causality requires going beyond functional analysis

  • need to know if intervention is responsible for change

• Example: sam & david fight everyday after school

  • mom institutes intervention at home to reduce fighting

    • baseline to assess current fighting behavior

    • intervention introduced

      • praise children for playing cooperativel

    • assess again

      • fighting is reduced

  • what caused fighting to decrease after intervention?

    • praise?

    • david recovered from illness

      • less irritable

    • sam made friends with lea next door

  • must rule out alternatives!

• ruling out alternatives

  • need experimental design to demonstrate the cause of behavior change

    • identify the variables that influence, control, or are responsible for behavior change

• want to demonstrate the functional relationship between the target behavior and the intervention

  • demonstrated when altering the experimental condition or contingency systematically changes behavior

• many different choice

  • single-case experimental designs

  • can be used with groups of individuals or with an individual

Basic Characteristics of Single-Case Designs: Continuous Assessment

• inferences are usually made about the effects of the intervention by comparing different conditions presented to 1+ individuals over time

• performance must be observed repeatedly over time

  • before intervention

  • continuously during intervention

    • daily

    • multiple times each week

  • sometimes after intervention

• interventions’ effects are examined by observing the influence of treatment vs. no treatment in 1 subject

Basic Characteristics of Single-Case Designs: Baseline Assessment

• observe behavior for several days before intervention is implemented

  • called baseline phase

  • provides info about level of behavior before intervention begins

• 2 functions

  • describe existing level of performance

    • descriptive function: extent of client’s problem

  • basis for predicting the level of performance for the immediate future if intervention isn’t provided

    • predictive function

• must gather baseline data for several days

  • need a basis for making predictiongs about future behavior

  • need multiple days to accurately extrapolate what future behavior would be

• if treatment is effective, performance will differ from the projected level of baseline

Basic Characteristics of Single-Case Designs: Stability of Performance

• baseline data must be stable

  • can’t predict future behavior from baseline if its unstable

  • stability is characterized by little variability in performance

• instability

  • trend in data

  • variability in data

• trend, AKA slope

  • tendency for data to systematically increase or decrease over time

    • OK for behavior to change in opposite direction from desired behavior

      • example: aggression escalates

  • behavior changing in same direction → difficulty determining causality

    • but, may not be improving quick enough

• variability

  • fluctuation in subject’s performance over time

  • excessive variability within any phase can → problems in concluding causality

Single-Case Experimental Designs

• true experimental designs

  • ABAB: reversal designs

  • multiple-baseline designs

  • changing-criterion designs

Single-Case Experimental Designs - ABAB (reversal designs)

• family of designs

• observations are made over time for a given client

  • changes are made in the conditions the client is exposed to

  • conditions are often alternated

• ABAB

  • A - baseline condition

  • B - intervention condition

  • A - baseline condition

  • B - intervention condition

• intervention effects clear if

  • performance improves during B

  • performance reverts during A to baseline

• variations

  • ABA - minimum to provide strong experimental demonstration of causality

  • AB - much weaker, not a true experiment, can’t infer causality

  • AB1B2AB2 - used when og intervention isn’t as effective as hope, a second intervention (B2) is added

  • cant substitute noncontingent reinforcement for return to baseline

    • deliver consequences independent of behavior

    • can provide evidence it’s not the event, but the relationship of the event to the behavior that → change

• strength - isolates effects of the intervention

• limitations

  • requires reversion or approximation of baseline

    • behavior doesn’t always revert

  • withdrawal of treatment

    • may be unethical

    • especially in clinical situations

Single-Case Experimental Designs - Multiple-Baseline Designs

• demonstrate the effect of an intervention by showing that behavior change accompanies introduction of the intervention at different points in time

  • evaluate change across different baselines

  • intervention is introduced to the different baselines at different time points

  • look to see if change occurs when intervention is introduced for each baseline

• doesn’t require that the intervention be withdrawn

  • clinically more useful

• variations

  • different individuals - useful when you want to alter behavior in a group of individuals

  • different situations - example: class, playground

  • different settings - example: home, school

  • different time periods - example: morning, afternoon, evening

• strengths

  • doesn’t require a return to baseline

  • user-friendly

  • good for clinical applications

  • can strengthen/alter intervention before implementation at each baseline

• limitations

  • no specificity regarding # of baselines needed

    • minimum - 2

  • must see change only when treatment is implemented

    • intervention may change other behaviors

    • why? because behavior is often interrelated

Single-Case Experimental Designs - Changing-Criterion Designs

• demonstrate effect of the intervention by showing that behavior matches a criterion for performance that is set for either reinforcement or punishment

• as the criterion is changed, behavior increases/decreases to match the criterion

• a causal relation is demonstrated if the behavior matches the constantly changing criterion

• variations

  • include a baseline phase

  • include a phase in which criterion is mad less stringent

• strengths

  • good for difficult terminal responses

  • good for behaviors where gradual progress is expected

• limitations

  • behavior may fall far under criterion is introduced

    • 25 cig/d → 10 cig/day

  • may take a long time

Group Designs - Basic Features

• need at least 2 groups

  • experimental group

  • control group

• compare rates in experimental group to control group

  • if significantly different, intervention is effective

• example: parents want to increase child’s study time

  • manipulation

    • experimental: parents learn to reinforce study behavior

    • control: no intervention

  • measure

    • do children in the exp group spend more time studying?

  • assessment is usually completed before and after intervention

    • in single-case designs, behavior is assessed during the intervention

  • in group research, differences between groups at treatment’s end are used to draw conclusions about effectiveness

    • need to ensure groups are similar to begin with

    • employ random selection and random assignment

Evaluating the data: has the behavior reliably changed?

• group designs: use statistics

  • statistics determine whether the differences are reliable

  • are they likely to be due to the different conditions the groups received?

• single-case designs: baseline vs. intervention performance

  • usually don’t use statistics

  • whether a systematic change is evident is inferred from the design and the pattern of the data across phases

  • what criteria to use?

• Characteristics of the data to decide if behavior changes are reliable based on graphical display

  • change in means

    • mean rate of the behavior shows a change from phase to phase in the expected direction

  • change in slope

    • direction of the slope changes from phase to phase

      • baseline - horizontal line

      • intervention - non-horizontal line

  • shift in level

    • when one phase changes to another, a level refers to the change in behavior from the last day of one phase and the first day of the next phase

      • break in the data

Is the change important?

• just because a change occurred doesn’t mean it’s important

• does the change make a difference in the person’s life?

  • clinical significance: practical or applied value or importance of the intervention’s effects

  • effects should be large enough to be of practical value or have impact on the everyday lives of those who receive the intervention, as well as those in contact with them

  • often involves returning client to normative level of functioning