Extinction & Stimulus Control Notes Chpt.8 E2

Extinction and Stimulus Control

Chapter Outline

• Extinction and Stimulus Control.

• Peak Shift Effect.

• Multiple Schedules and Behavioral Contrast.

• Fading and Errorless Discrimination Learning.

• Applications of Stimulus Control.

8.1 Extinction

• Operant extinction: weakening of a previously reinforced behavior by non-reinforcement.

• Extinction is both a procedure (non-reinforcement) and a process (resulting decrease in response).

• Example scenario: Rat presses lever (R) for food (SR). If lever press (R) → No food, lever pressing declines as no food is given when lever is pressed.

• Example scenario: Child whines (R) for candy (SR). If whining (R) → No candy, whining declines as candy is not given when they whine.

• Ensure the withheld consequence is the actual reinforcer maintaining the behavior.

• Attention can be the actual reinforcer not candy.

• Extinction should only be used when the future strength of a behavior declines because the reinforcer for that behavior is removed.

• Decrease in strength of behavior that is not extinction include:

  • Positive Punishment (shock for lever press).

  • Abolishing Operation (satiation with free food).

  • Removing the lever; removing iPad.

Side Effects of Extinction

• Extinction Burst:

  • A temporary increase in the frequency and intensity of responding when extinction is first implemented.

  • Example: Rat pressing lever more rapidly/forcefully when food stops.

  • Candy machine analogy: Pressing the button repeatedly with more force.

• Increase in Variability:

  • Behavior becomes more variable.

  • Example: Rat pressing lever with different paws or sides.

  • Candy machine analogy: Varying button- pushing methods.

• Emotional Behavior:

  • Frustration and agitation may occur.

  • Example: Pigeon becoming agitated when key pecking stops producing food.

  • Candy machine analogy: Becoming upset when the machine doesn't deliver.

• Aggression:

  • Extinction procedures used to study aggressive behavior.

  • Example: Pigeon attacking another pigeon when key pecking is on extinction (frustration-induced aggression).

  • Road rage/air rage may be adult temper tantrums.

• Resurgence:

  • Reappearance of previously effective behaviors during extinction.

  • Example: Rat returning to a previously reinforced maze pattern.

  • Resembles regression.

  • Example: Someone ignored by their partner spending time at their parents' house b/c they are not getting reinforced in romantic relationship, returning to setting that's rich with reinforcement.

• Depression:

  • Depressive-like symptoms can occur.

  • Rats running down alleyway for food followed by open field test. Extinction on alleyway task led to increased activity in the open field test, and then decreased activity.

  • Klinger et al. (1974) noted low activity is a common symptom of depression.

  • Depression is also associated with loss of reinforcement.

  • A temporary period of depression after losing a major reinforcer is normal for disengagement.

• Side effects can be strengthened if the reinforcer is given in the midst of the side effects.

• Example: Bobbie gets candy after a tantrum, the behavior increases for the parent giving the candy.

Figure 8.1: Changes in rats’ activity level in an open field test as a function of extinction on a preceding straight-alley maze task.

Resistance to Extinction

• Resistance to extinction: the extent to which responding persists after an extinction procedure is implemented.

• High resistance: persistent responding.

• Low resistance: quick disappearance of the response.

Figure 8.2: Two hypothetical extinction curves. Following an initial period of reinforcement at the start of the session, the extinction procedure is implemented. This results in a brief extinction burst, followed by a decline in responding. The decline is more gradual in the top example than in the bottom example and hence illustrates greater resistance to extinction.

Factors affecting resistance to extinction:

• Schedule of Reinforcement:

  • Partial reinforcement effect: intermittent schedules lead to greater resistance to extinction than continuous reinforcement.

  • $FR 10$ extinguishes slower than $FR 1$.

  • $FR 100$ extinguishes slower than $FR 10$.

  • Variable interval (VI) and variable ratio (VR) schedules produce strong resistance.

  • $VI 30-sec$ produces greater resistance compared to $FI 30-sec$.

  • $VR 100$ produces greater resistance compared to $FR 100$.

  • The less frequent/ predictable reinforcement, the longer it takes to "discover" reinforcement is unavailable.

  • Greater contrast between CRF schedule and extinction than between VR 100 schedule and extinction.

  • Dog begging example.

    • Partial reinforcement effect can result in certain types of annoying or maladaptive behaviors that are difficult to eliminate.

    • Dog nagging for food at the dinner table paradoxically increases the nagging behavior.

    • Continuously reinforced is less resistant to extinction.

    • Reinforce each instance of begging for several days then implement extinction to rapidly decrease the nag.

• History of Reinforcement:

  • More reinforcers received for a behavior lead to greater resistance to extinction.

  • Lever pressing extinguishes more rapidly with 10 reinforcers than with 100.

  • Whining for candy stops quicker if the habit is recent.

  • "Nip it in the bud".

  • Furomoto (1971) shows maximum resistance after 1,000 reinforcers.

• Magnitude of the Reinforcer:

  • Large-magnitude reinforcers sometimes result in more resistance.

  • Large food pellet vs small food pellet.

  • Preferred vs. less preferred food item.

  • Feed the dog small bites of less preferred morsels.

  • Smaller reinforcers sometimes result in greater resistance (Ellis, 1962).

• Degree of Deprivation:

  • Greater deprivation leads to greater resistance.

  • Slightly hungry rat stops lever pressing quicker than a very hungry rat.

  • Feed the dog before the meal.

• Previous Experience with Extinction:

  • Alternating extinction/reinforcement sessions leads to quicker extinction during subsequent exposures.

  • Rat learns that no reinforcement is likely after the start of an extinction session.

  • Breakups become more manageable after numerous experiences.

• Distinctive Signal for Extinction:

  • Extinction is quicker with a stimulus that signals its onset.

  • Discriminative stimulus for extinction ($S^\triangle$).

Spontaneous Recovery

• The reappearance of an extinguished response after a rest period.

• Extinguished behavior is not permanently eliminated.

• Rat commences lever pressing again after extinction. But it will likely be weaker and extinguish more quickly given that we continue to withhold reinforcement.

• Each recovery is weaker and more readily extinguished.

• Skinner (1950) says it is a function of discriminative stimuli ($S^D$) associated with session start.

Figure 8.3: Graph of hypothetical data illustrating spontaneous recovery across repeated sessions of extinction.

Differential Reinforcement of Other Behavior (DRO)

• Enhance extinction by reinforcing a replacement behavior.

• Reinforcement of any behavior other than the target behavior.

• More effective because the target behavior is weakened by (1) lack of reinforcement and (2) reinforcement of alternative behaviors.

• Reduces deprivation and side effects.

• Functional communication training: teaching a child to communicate their needs appropriately.

• Provide positive reinforcement for appropriate behavior during attempted extinguished of unwanted behavior. Miltenberger (2012).

8.2 Stimulus Control

• When a behavior has been consistently reinforced in the presence of a certain stimulus, that stimulus will begin to affect the probability of the behavior.

• Discriminative stimulus ($S^D$): stimulus that signals the availability of reinforcement i.e the likelihood of the behavior occurring

• The $S^D$ does not elicit behavior in the way that a CS elicits a CR

• A Tone is played (CS) for response of salivation, a light ($S^D$) signals the availability of food for lever pressing.

• Behavior is under stimulus control: presence of discriminative stimulus reliably affects the probability of the behavior.

• Example of stimulus control:

  • If a 2000-Hz tone ($S^D$): Lever press (R) → Food (SR) rat learns to press the lever only in the presence of the tone.

  • Ringing phone.

  • Traffic lights.

  • Saying "Thank you" leads to the response of "you are welcome".

  • Notification on cell phone.

Stimulus Generalization and Discrimination

• Stimulus generalization: tendency for an operant response to be emitted in the presence of a stimulus that is similar to an $S^D$.

• More similar stimulus = more response.

• Rat learns to press a lever for food whenever it hears a 2000 HZ tone present rat with a series of tones that vary in pitch will find that it also presses the lever in the presence of these other tones, particularly in the presence of a tone that is similar to the original $\text{S}^D$.

• Generalization gradient: strength of responding in the presence of similar stimuli.

• Steep gradient-rate of responding drops sharply as stimuli become increasingly different from the $S^D$.

• Flat gradient-responding drops gradually as the stimuli become increasingly different from the $S^D$ i.e there is more generalization.

• Stimulus discrimination: emitting an operant response more in the presence of one stimulus than another.

Figure 8.4: Two hypothetical generalization gradients depicting rate of lever pressing in the presence of tones that vary in pitch between 1200 and 2800 Hz (“Hertz” is the number of sound waves per second generated by a sound source). In both examples, tones that are more similar to the original SD (a 2000-Hz tone) are associated with stronger responding. However, generalization is much greater in the bottom gradient, which is relatively flat, than in the top gradient, which is relatively steep.

• More generalization = less discrimination and vice versa.

• Discrimination training: reinforcing responses in the presence of one stimulus (the $S^D$) and not another.

• Discriminative stimulus for extinction ($S^\triangle$): stimulus that signals the absence of reinforcement.

• Discrimination training procedure:

  • 2000-Hz Tone ($S^D$): Lever press (R) → Food (SR).

  • 1200-Hz Tone ($S^\triangle$): Lever press (R) → No food.

  • The rat will learn to press the lever in the presence of 2000-Hz tone and not in the presence of 1200-Hz tone.

• Matching-to-sample procedure.

The Peak Shift Effect

• The peak of a generalization gradient following discrimination training will shift from the $S^D$ to a stimulus that is further removed from the $S^\triangle$.

• Exception to the general principle i.e strongest response should be in the presence of the originial $S^D$.

• Fanciful example from transcript: Mr. Shallow seeks out women who are even more extroverted than his last partner i.e even more extreme than $\text{S}^D$.

• Subjects respond in terms of the relative values of stimuli

• The rat learns that a higher-pitched tone indicates food and a lower-pitched tone indicates no food.

• Despite discrimination training, the $S^D$ is still somewhat similar to the $S^\triangle$ and has acquired some of its negative properties.

Figure 8.5: Illustration of a peak shift effect following discrimination training. Prior to discrimination training (top panel), the gradient is relatively flat. Following discrimination training (bottom panel), in which a 1200-Hz tone has been established as an S