Module 8:Schedule of Reinforcement

Foundations of Schedules of Reinforcement

  • Definition: A schedule of reinforcement is a rule describing a contingency of reinforcement, specifically the response-contingent reinforcement relationship (the requirement that an individual must engage in a specified response to access reinforcement).

  • Boundaries of Reinforcement Schedules:

    • Continuous reinforcement (CRFCRF) and extinction (EXTEXT) establish the boundary conditions for all other reinforcement schedules (100%100\% vs. 0%0\%).

    • Continuous Reinforcement (CRFCRF): Delivers reinforcement for 100%100\% of occurrences of the target behavior. This is an FR 1FR\,1 schedule and is utilized during the initial stages of learning to acquire and strengthen new behaviors.

    • Extinction (EXTEXT): According to Cooper, extinction is technically classified as a schedule of reinforcement where the schedule value is 00. Reinforcement is provided for 0%0\% of occurrences of the target behavior (no reinforcement is delivered for any occurrence).

    • Intermittent Reinforcement: Encompasses all schedules existing between extinction and continuous reinforcement (e.g., reinforcing approximately 50%50\% or 60%60\% of responses). Intermittent schedules are used to maintain behaviors that have already been established in the repertoire.

  • Behavioral Persistence and Reinforcement Mechanics:

    • The functional definition of reinforcement is to increase the future frequency or occurrence of a behavior.

    • For a behavior to maintain over time, it must continue to access reinforcement; if a behavior completely ceases to access reinforcement, its frequency will decline until it stops occurring.

  • Maintenance and Generality in Applied Behavior Analysis (ABAABA):

    • Maintenance refers to a lasting change in behavior that persists after the intervention program has been terminated. Practitioners must ensure behavioral gains endure post-treatment.

    • Generality is one of the seven core dimensions of ABAABA, requiring behavior change to maintain across settings, individuals, and time.

    • Naturally Occurring vs. Contrived Reinforcers:

      • Naturally occurring reinforcers are environmental stimuli and activities already present in the natural setting that function as reinforcement.

      • Contrived reinforcers are artificial stimuli arranged and delivered by an outside person (e.g., a practitioner delivering edibles during a session).

      • If an intervention relies exclusively on contrived reinforcers and that specific person departs (e.g., changes jobs), the reinforcement ceases, leading to extinction and the loss of the behavior.

      • Interventions should incorporate behaviors that access naturally occurring contingencies (e.g., teaching a child who loves carrots to mand for carrots when carrots are readily present in his natural environment), allowing behavioral maintenance without practitioner presence.

Basic Intermittent Schedules of Reinforcement

  • Core Classification Dimensions:

    • Ratio vs. Interval Schedules:

      • Ratio Schedules: Delivery of reinforcement is contingent solely on a specified number of responses emitted before reinforcement is provided (e.g., needing to buy 1515 drinks at Starbucks to earn a free drink, or a student completing 1515 math problems).

      • Interval Schedules: Delivery of reinforcement is contingent upon the lapse of a specified duration of time before a response can produce reinforcement. Time alone does not deliver reinforcement; the target response must occur after the interval has elapsed.

    • Fixed vs. Variable Schedules:

      • Fixed Schedules: The response ratio or elapsed time requirement remains constant and invariant across reinforcement deliveries.

      • Variable Schedules: The response ratio or elapsed time requirement varies across deliveries, fluctuating unpredictably around a predetermined average value. Because variable schedules are unpredictable, they are superior at maintaining behavior over time.

  • The Four Basic Intermittent Schedules:

    • Fixed Ratio (FRFR)

    • Variable Ratio (VRVR)

    • Fixed Interval (FIFI)

    • Variable Interval (VIVI)

Fixed Ratio (FR) Schedules

  • Definition: An FRFR schedule requires the completion of a set, invariant number of responses for the delivery of reinforcement.

  • Notation: Denoted as FR nFR\,n, where nn indicates the exact number of responses required (e.g., FR 5FR\,5 requires 55 responses, FR 15FR\,15 requires 1515 responses).

  • Examples:

    • Damian earns reinforcement each time he reads 55 pages in his book (FR 5FR\,5).

    • Starbucks rewards requiring the collection of 1515 stars to receive a free drink (FR 15FR\,15).

  • Pattern of Responding on Cumulative Records:

    • Post-Reinforcement Pause (PRPPRP): A transient pause in responding that occurs immediately after reinforcement delivery. On a cumulative record, reinforcement delivery is indicated by a slash or dash mark, followed by a flatline segment indicating no responding during the pause.

    • Ratio Run: Following the pause, the individual emits responses rapidly with little or no hesitation between responses until the ratio requirement is completed.

    • Rate Maximization: Rapid responding directly maximizes the frequency of reinforcement delivery; completing responses faster yields reinforcement faster.

    • Ratio Size Effects: Larger ratio requirements yield higher rates of response during the run, but they also produce longer post-reinforcement pauses. For example, kicking 1515 to 2020 soccer balls into a goal produces a short pause, whereas requiring 100100 soccer kicks or completing 100100 math problems produces a substantially longer pause due to fatigue and the known magnitude of the upcoming requirement.

Variable Ratio (VR) Schedules

  • Definition: A VRVR schedule requires the completion of a variable number of responses to access reinforcement, with the requirement fluctuating unpredictably around a specified average value.

  • Notation: Denoted as VR nVR\,n, where nn is the average response requirement.

  • Example: Damian earns reinforcement on average after reading every 55 pages (VR 5VR\,5), where individual requirements vary unpredictably across iterations (e.g., 33, 1010, 77, 22, and 33 pages, averaging 55 pages).

  • Pattern of Responding on Cumulative Records:

    • Produces high, consistent, and steady rates of responding without a post-reinforcement pause.

    • Absence of Pause: Because reinforcement delivery is unpredictable, the individual cannot foresee when the next reinforcer will occur; any given response could contact reinforcement, prompting sustained responding.

    • Maintenance Efficacy: Variable ratio schedules are the most effective basic schedule for maintaining behaviors over extended periods and resisting extinction.

    • Resilience to Thinning: When schedules are gradually thinned, individuals maintain responding even under exceptionally high ratio requirements (e.g., a VR 200VR\,200 schedule where reinforcement is accessed periodically after dozens of responses).

    • Real-World Prototype: Slot machines operate on variable ratio schedules. They require a high number of monetary responses to pay out unpredictably, maintaining high and consistent rates of play.

Fixed Interval (FI) Schedules

  • Definition: An FIFI schedule delivers reinforcement for the first response emitted following the lapse of a fixed, predetermined duration of time.

  • Notation: Denoted as FI tFI\,t, where tt is the duration of the time interval (e.g., FI 5 minutesFI\,5\text{ minutes}).

  • Contingency Mechanics: Responses occurring during the elapsing interval have no effect on reinforcement delivery. The reinforcer is only produced by the first response emitted after the interval has elapsed.

  • Examples:

    • Damian answering the first question in class following the passage of a 5 minute5\text{ minute} interval (FI 5FI\,5).

    • Potty training where a child is reinforced for the first instance of voiding in the toilet after an interval of 10 minutes10\text{ minutes} has elapsed, irrespective of how many times voiding occurred during the interval.

  • Pattern of Responding on Cumulative Records:

    • Post-Reinforcement Pause: A distinct pause follows reinforcement delivery during the early portion of the interval.

    • Accelerated Rate (FIFI Scallop): As the interval draws closer to its conclusion, the individual increases response frequency, generating a characteristic scalloped curve on a cumulative record.

    • Mnemonic: The term FIFI can be associated with fish, which has scales, or ocean scallops.

    • Overall Rate: Produces a slow to moderate overall rate of response.

    • Real-World Analogy: Flash-sale shopping websites (such as Rue La La or HauteLook) opening heavily discounted designer goods at exactly 11:00 AM EST11:00\,\text{AM EST}. Refreshing the browser at 9:00 AM9:00\,\text{AM} yields no consequence, but response rates for page refreshing dramatically accelerate at 10:55 AM10:55\,\text{AM} and 10:56 AM10:56\,\text{AM}, peaking at 11:00 AM11:00\,\text{AM}.

Variable Interval (VI) Schedules

  • Definition: A VIVI schedule delivers reinforcement for the first response following the lapse of a variable, unpredictable interval of time that averages a specified duration.

  • Notation: Denoted as VI tVI\,t, where tt is the average duration of the interval.

  • Examples:

    • Damian receiving reinforcement for answering a question after an average interval of 5 minutes5\text{ minutes} has elapsed (e.g., actual intervals spanning 33, 1010, 77, 22, and 3 minutes3\text{ minutes}).

    • Potty training reinforced for the first void after variable intervals averaging 10 minutes10\text{ minutes} (e.g., 88, 1212, 55, 1515, and 10 minutes10\text{ minutes}).

  • Pattern of Responding on Cumulative Records:

    • Produces a steady, consistent rate of response with minimal or no post-reinforcement pauses.

    • Absence of Pausing: Because the interval duration varies unpredictably, reinforcement could become available at any point, maintaining steady engagement.

    • Interval Magnitude Relationship: The larger the average interval duration, the lower the overall steady rate of response emitted.

    • Real-World Analogy: If Rue La La opened its shopping deals daily at completely unpredictable times between 10:00 AM10:00\,\text{AM} and 11:00 AM11:00\,\text{AM}, checking behavior would remain steadily elevated without pauses throughout that temporal window.

Limited Hold Procedures

  • Definition: A limited hold (LHLH) is a finite period of time placed upon an interval schedule (FIFI or VIVI) during which the target response must occur after the interval has elapsed in order to produce reinforcement.

  • Clinical Rationale: In an unconstrained interval schedule, reinforcement remains available indefinitely after the interval elapses until the response is emitted. A limited hold restricts this window of availability.

  • Operational Dynamics:

    • If the target response is emitted within the limited hold window, reinforcement is delivered and the primary interval restarts.

    • If the target response is not emitted before the limited hold expires, reinforcement is forfeited and the primary interval restarts immediately without delivery of the reinforcer.

  • Case Example: An FI 10 minuteFI\,10\text{ minute} schedule combined with a 5 minute5\text{ minute} limited hold (FI 10 min LH 5 minFI\,10\text{ min LH }5\text{ min}) for toilet training:

    • The toddler waits for the 10 minute10\text{ minute} interval to pass.

    • Once the timer sounds, the 5 minute5\text{ minute} limited hold starts.

    • If the toddler voids within those 5 minutes5\text{ minutes} (e.g., at minute 22), reinforcement is delivered and the 10 minute10\text{ minute} interval resets.

    • If the toddler does not void during the 5 minute5\text{ minute} hold, no reinforcement is given and the 10 minute10\text{ minute} interval resets.

Schedule Thinning and Ratio Strain

  • Procedures for Thinning Schedules of Reinforcement:

    • Thinning is the systematic progression from a dense schedule of reinforcement (frequent reinforcement delivery) to a leaner schedule (less frequent delivery).

    • Procedure 1: Gradually increasing the response ratio requirement (e.g., increasing successful soccer goals from 55 to 88, 1010, 2020, and then 3030 per day).

    • Procedure 2: Gradually increasing the duration of the time interval.

    • Facilitating Transitions: Utilizing clear verbal instructions to communicate schedule parameters helps facilitate smooth transitions when thinning schedules.

  • Ratio Strain:

    • Etiology: Occurs when schedules are thinned too rapidly, when response requirements become excessively high relative to the value of the reinforcer, or when the response requirement exceeds the individual's physiological capabilities (e.g., requiring an individual to kick a soccer ball 150150 or 2,0002{,}000 times).

    • Characteristics: Manifests as avoidance, aggression, and unpredictable, prolonged pauses in responding.

  • Remediation of Ratio Strain:

    • Step 1: Immediately revert to the last successful schedule of reinforcement where responding was stable.

    • Step 2: Re-establish behavioral control under that dense schedule.

    • Step 3: Modify the schedule again using smaller, more gradual increases in the ratio requirement.

Variations on Basic Intermittent Schedules

  • Differential Reinforcement of Specific Rates of Responding (Ratio Variations):

    • Interventions used when target behaviors are problematic due to their rate (occurring too frequently or not frequently enough).

    • Differential Reinforcement of High Rates (DRHDRH):

      • Mechanism: Reinforcement is contingent on response rates that are higher than a predetermined criterion.

      • Objective: Produces an accelerated, higher rate of responding.

      • Examples: Providing reinforcement when a client asks 33 questions in class per day, then increasing the criterion to 55 questions; increasing daily SAFMEDS memorization criteria from 22 cards to 33 cards.

    • Differential Reinforcement of Low Rates (DRLDRL):

      • Mechanism: Reinforcement is delivered when the behavior occurs at a rate lower than a predetermined criterion.

      • Objective: Decreases high-rate behaviors without eliminating them entirely.

      • Example: Reinforcing a student who asks 2020 questions per day when they ask 1515 or fewer questions, then progressively lowering the criterion to 1212 or fewer questions across subsequent sessions.

    • Differential Reinforcement of Diminishing Rates (DRDDRD):

      • Mechanism: Reinforcement is provided at the end of a predetermined time interval when the number of responses is less than a criterion that is systematically decreased across consecutive time intervals based on performance.

      • Example: Requiring fewer than 55 responses every 5 minutes5\text{ minutes}, then systematically reducing the threshold to fewer than 44 responses every 5 minutes5\text{ minutes}, and subsequently fewer than 33 responses every 5 minutes5\text{ minutes}.

  • Lag Schedules of Reinforcement:

    • Mechanism: Reinforcement is contingent upon a response differing in some predetermined way (e.g., topography, vocal response, or sequence) from one or more immediately preceding responses.

    • Objective: Systematically increases response variability and expands behavioral repertoires.

    • Lag 11 Schedule: The emitted response must differ from the single immediately preceding response that produced reinforcement.

      • Letter Teaching Example: An instructor asks, "Tell me a letter," and the student responds "A" (reinforced). If prompted again and the student says "A", reinforcement is withheld. If the student says "G", reinforcement is delivered. After "G", the student can say "A" again or provide another letter.

    • Lag 22 Schedule: The response must differ from the previous two responses that contacted reinforcement.

      • Letter Teaching Example: If the sequence is "A" (reinforced), followed by "G" (reinforced), a subsequent response of "A" is not reinforced; the student must provide a novel letter such as "R" to earn reinforcement.

    • Applied Application: Expanding verbal repertoires beyond stereotyped responses (e.g., if a student consistently names "bird" when asked to identify flying animals, a lag schedule withholds reinforcement for "bird" to reinforce alternative flying animals).

  • Progressive Schedules of Reinforcement:

    • Mechanism: Systematically thins reinforcement requirements within a session across successive opportunities, independent of the participant's behavior.

    • Objective: Identifies the individual's breaking point—the threshold where the response requirement becomes too high for the reinforcer to maintain performance.

    • Example: A student completing math problems is reinforced on an FR 1FR\,1 schedule, which is thinned within-session to FR 2FR\,2, FR 3FR\,3, FR 5FR\,5, and FR 10FR\,10, observing the point at which responding breaks down.

    • Ethical and Clinical Caution: Exposure to extreme ratio requirements in progressive schedules can be aversive; practitioners should not implement progressive schedules without clear, offsetting clinical benefits.

Compound Schedules of Reinforcement: Principles and Classification

  • Definition: A compound schedule consists of two or more basic schedules of reinforcement (e.g., FRFR, VRVR, FIFI, VIVI, DRLDRL, DRHDRH, CRFCRF, or EXTEXT) combined in a specific configuration.

  • Component Schedules: The individual constituent schedules that make up the compound schedule.

  • Discrimination Classifications:

    • Discriminated Compound Schedules: A distinct discriminative stimulus (SDS^D) is present and correlated with each component schedule, signaling which schedule is active.

    • Nondiscriminated Compound Schedules: No discriminative stimulus is present to signal the active component schedule.

  • The Five Primary Compound Schedules:

    • Concurrent Schedules (conc\text{conc}

    • Multiple Schedules (mult\text{mult}

    • Mixed Schedules (mix\text{mix}

    • Chained Schedules (chain\text{chain}

    • Tandem Schedules (tand\text{tand}

Concurrent Schedules of Reinforcement

  • Mechanics:

    • Two or more schedules of reinforcement operate simultaneously and independently for two or more distinct behaviors.

    • An SDS^D is correlated with each component schedule.

    • The individual can freely alternate between and allocate responding across the available schedules.

  • Chore Example:

    • Jana performs chores on Sundays with simultaneous access to folding laundry on an FR 10FR\,10 schedule and washing dishes on an FR 3FR\,3 schedule.

    • Reinforcement is delivered for completing either component schedule independently (e.g., washing 33 dishes yields a reinforcer; folding 1010 items yields a reinforcer).

    • Jana can freely allocate behavior: she can wash 33 dishes (earning a reinforcer), fold 44 items of laundry, return to wash 33 more dishes (earning another reinforcer), and then complete the remaining 66 laundry items to earn the laundry reinforcer.

  • Clinical Applications in ABAABA:

    • Reinforcer Assessments: Evaluates relative preference and reinforcer efficacy by observing where an individual allocates responding. For example, presenting two identical math worksheets at separate desks—one providing access to an iPad upon completion and the other providing access to Play-Doh. If responding is allocated predominantly to the Play-Doh table, Play-Doh is demonstrated to possess higher reinforcing potency.

    • Interventions to Shift Behavioral Allocation: Practitioners increase the magnitude or quality of reinforcement on one alternative to bias responding away from problematic or unsafe alternatives.

    • Clinical Case Example: A client spent entire evenings and nights standing near the kitchen stove (a major safety concern). A concurrent schedule was implemented: outside the kitchen, the client had unrestricted access to highly preferred items and activities; inside the kitchen, only a single low-preference item was available. Both areas remained freely accessible, but the concurrent reinforcement disparity successfully biased the client's responding toward remaining outside the kitchen in safe spaces.

Multiple vs. Mixed Schedules of Reinforcement

  • Shared Structural Characteristics:

    • Both multiple and mixed schedules consist of two or more component schedules that alternate (across days, sessions, or successive intervals).

    • Only one component schedule is in effect at any given point in time.

    • Reinforcement is delivered whenever the response requirements of the active schedule are satisfied.

  • Multiple Schedules (mult\text{mult}):

    • A discriminated schedule featuring an explicit SDS^D correlated with each component schedule.

    • Example: Jana's laundry chores are distributed between red and blue bins, assigned randomly across days or sessions:

      • Red bin days operate on an FR 10FR\,10 schedule (requires folding 1010 items).

      • Blue bin days operate on an FR 3FR\,3 schedule (requires folding 33 items).

      • The bin color serves as the SDS^D, signaling the schedule requirement in effect (mult FR 10 FR 3mult\,FR\,10\,FR\,3).

  • Mixed Schedules (mix\text{mix}):

    • A nondiscriminated schedule identical to a multiple schedule, but lacking an SDS^D.

    • Example: Jana's laundry is placed in a single uniform bin, alternating unpredictably between an FR 10FR\,10 and an FI 5 minuteFI\,5\text{ minute} schedule (mix FR 10 FI 5mix\,FR\,10\,FI\,5). Reinforcement is delivered either after 1010 items are folded or for the first item folded after 5 minutes5\text{ minutes} has elapsed, but no stimulus signals which contingency is operating.

Chained vs. Tandem Schedules of Reinforcement

  • Shared Structural Characteristics:

    • Both chained and tandem schedules require the completion of two or more component schedules in a specific, invariant, sequential order.

    • Sequential Dependency: The response requirement of the first component schedule must be met before advancing to the second component schedule.

    • Terminal Reinforcement: Reinforcement is delivered only upon completion of all component schedules in the chain; intermediate components do not deliver terminal reinforcement.

  • Chained Schedules (chain\text{chain}):

    • A discriminated sequence where a distinct SDS^D is correlated with each component schedule in the sequence.

    • Example: On Sundays, Jana is first presented with dirty dishes operating on a VR 3VR\,3 schedule (washing an average of 33 dishes). Completing this requirement yields no reinforcement, but prompts her parents to present the laundry bin (SDS^D) with an FR 10FR\,10 schedule (folding 1010 items). Completing the laundry fulfills both components and produces terminal reinforcement (chain VR 3 FR 10chain\,VR\,3\,FR\,10).

  • Tandem Schedules (tand\text{tand}):

    • A nondiscriminated sequence identical to a chained schedule, but lacking correlated SDS^Ds to signal component transitions.

    • Example: A schedule designated as tand FR 12 FI 4 minutestand\,FR\,12\,FI\,4\text{ minutes}. Jana must complete 1212 chore responses (FR 12FR\,12) followed by emitting the first chore response after a 4 minute4\text{ minute} interval elapses (FI 4FI\,4) to receive reinforcement, without any external stimuli signaling the completion of the FR 12FR\,12 or the onset of the FI 4FI\,4 interval.

Diagnostic Decision Framework and Mnemonics for Compound Schedules

  • Two-Step Diagnostic Decision Algorithm:

    • Step 1: Determine whether a sequence exists.

      • Question: Does the individual have to complete multiple response requirements in a mandatory sequence before accessing reinforcement?

      • If Yes: The schedule is either Chained or Tandem.

      • If No: The schedule is Concurrent, Multiple, or Mixed.

    • Step 2: Determine whether a discriminative stimulus (SDS^D) is present.

      • Question: Is there an environmental cue signaling which schedule is active or signaling transitions between components?

      • If Yes (Discriminated): The schedule is Concurrent, Multiple, or Chained.

      • If No (Nondiscriminated): The schedule is Mixed or Tandem.

  • Associative Mnemonics:

    • Global Mnemonic: Two individuals riding a tandem bicycle with chains along a concurrent path while eating M&Ms.

    • Pairing Rules:

      • Tandem bicycles have chains: Links Tandem and Chained schedules together as sequential arrangements.

      • M&Ms: Links Multiple and Mixed schedules together as alternating arrangements.

    • Conceptual Visualizations:

      • Concurrent Schedules: Resemble parallel roads running side-by-side; an individual can freely transition back and forth between roads at any point.

      • Multiple vs. Mixed Schedules: Multiple schedules resemble individually labeled chip bags (e.g., Lay's vs. Fritos), where packaging functions as an SDS^D signaling contents. Mixed schedules resemble a mixed bag of nuts or Chex Mix, where items are retrieved without predictive cues.

      • Chained vs. Tandem Schedules: Chained schedules resemble a linked chain where each link is a visible SDS^D leading toward ringing a bell at the end (terminal reinforcement). Tandem schedules require working through the identical sequence, but without visible links or signaling stimuli.