Ch 8 - Extinction & Stimulus Control

Extinction

The process whereby a conditioned response decreases or disappears due to the cessation of reinforcement or the absence of the conditioned stimulus

• decreasing future probability of a response via nonreinforcement

• removing reinforcer/stop presenting it, nothing to wor for/motivate to continue

What is key to determine about reinforcers?

The nature and timing of the reinforcers must be analyzed to understand their effect on behavior

• need to make sure you have the right stim

• ex→ tantrum (R) → Food (S^R) & ipad (S^R), actual reinforcer is parents’ attention

Side Effects & Complications of Extinction

extinction burst, emotional behavior, inc in variability, resurgence

Extinction Burst

orgs might show an inc response rate at first (this should be working)

• side effect/complication from extinction

Emotional Behavior

represented in behavior, frustration, aggression, depression

• side effect/complication from extinction

Increase in Variability from Extinction

trying new behaviors to see what if anything has changes

• ext burst dec = X learning, jus stop responding

• side effect/complication from extinction

Extinction & Resurgence

return of past behaviors that worked even some time ago\

• org trying to find new solution = new learn

• side effect/complication from extinction

Resistance to Extinction

failure to modify a response schedule during extinction trials

• X providing reinforcer = org continues to respond

• orgs showing high resistance will persist w/ responding for greater time

• org w/ low resistance = rapid decline in response rate, quickly stopping

Sources of Resistance

schedule, history, magnitude of reinforcer, degree of deprivation, prev experience, discriminating stim

Source of Resistance: Schedule

lean & variable schedules show much greater resistance to extinction

• knows it has delay/will come later

Source of Resistance: History

behaviors that have been reinforced over a long time show greater resistance than new behaviors

Source of Resistance: Magnitude of Reinforcer

larger or highly appetitive reinforcers show greater resistance

Source of Resistance: Degree of Deprivation

orgs that are sated will more quickly extinguish behavior

Source of Resistance: Previous Experience

if behavior has extinguished in the past, will extinguish more quickly

Source of Resistance: Discriminating Stimulus

presence of a discriminating stimulus indicating extinction

How can resistance be controlled?

continue responding w/ reinforcer = frustrating & bothersome

How can resistance be increased?

Resistance can be increased by providing intermittent reinforcement, which makes the behavior more persistent despite the absence of the reinforcer.

• inc resistance = give in/give desired reinforcer, doing in a way that benefits extra causes

• more lean& variable schedule

• tells extinction burst works

How can resistance be decreased?

Resistance can be decreased by consistently applying extinction procedures, ensuring that the reinforcer is no longer available, which reduces persistent behavior.

• inc satiety (feed dog at dinner) → dec food drive

• dec food given slowly over time

• make food less appetitive reward

Spontaneous Recovery

is the re-emergence of previously extinguished behavior after a period of non-reinforcement, suggesting that the learned behavior is not entirely forgotten.

• organisms will sometimes adopt old schedules

• adaptive → X know if will return, there was a time beh got a response so trying again

• X want to entirely lose beh (ex → stop foraging in winter, X forget for summer)

Skinner & Spontaneous Recovery

thought it’s because testing enviros provided a discriminative stimulus (S^D) that indicates testing will occur

• in different environments that signals the possibility of reinforcement.

Spontaneous Recovery: Implications for Environment & Human Behavior

Spontaneous recovery highlights how past reinforcement experiences can influence behavior in new environments, reminding us that learned behaviors may resurface even when reinforcement is absent. This can be significant in understanding behavioral adaptations in varying contexts.

• ex- rehab → stop but when go back to of enviro = go back to prev beh, easier to extinguish out of enviro

How does replacing behaviors improve success?

differential reinforcement of other behavior (DRO)

• training paradigm → animals & humans

Differential Reinforcement of Other Behavior (DRO)

can improve extinction success by simult reinforcing a new behavior

• ex- behavior of rowdy dog

  • behavioral excitement (S^D):rough play (R) → owner attention (S^R)

  • behavioral excitement (S^D):rough play (R) → none (X reduce behavior)

  • behavioral excitement (S^D):play at dog park (R) → owner attention (S^R)

  • ^^ new discrim stim, acceptable enviro, reinforce as “new” R based on new enviro

Discriminatory Stimulus (S^D)

stim that tells an organism when responding will result in in a reinforcer, evokes response (animal has to choose to do)

• guide behavior in presence of stim

• light (S^D):lever press (R) → food (S^R)/reward

• phone ring (S^D):answer phone (R) → conversation (S^R)/reward

  • in both examples, S^D lets them know they can choose to do beh & will get rewarded

Discriminative Stimulus for Extinction (SΔ)

indicates a behavior won’t be reinforced

• coming after beh is done, beh only evoked

• might as well X do it, X reinforcement coming

• affect beh probability, X eliciting an unconscious response

Stimulus Generalization

tendency for a high response prob for stimuli similar to the S^D

• tone lets know = if press bar, will get something

Stimulus Discrimination

tendency of a diff in responding rate b/w stimuli

• learned ability to differentiate between similar stimuli, leading to distinct responses. It allows an organism to respond differently based on the presence or absence of a specific discriminative stimulus.

Generalization Gradient

how steeply generalization does/X fall off as stim changes

Discrimination Training

is the process of reinforcing a subject's ability to differentiate between similar stimuli. It involves presenting a specific discriminative stimulus while withholding reinforcement for other similar stimuli.

• Sdelta & S^D being used

• ex → light/color = press & get food, dark/dark colors = X get food/less response

How is discrimination training seen in normal life?

key part of social interactions, altering behavior to best fit environment

• teaching children behavior discrimination → right place & time for things

• traffic lights, when to stop/go

• behavior will change b/w being at a party, being in a church, or being at work

Stimulus Control

org adapting to a schedule based on an S^D &/or Sdelta is under this control

• when behavior is triggered by specific stimuli. It is the degree to which a response is influenced by the presence or absence of discriminative stimuli.

Peak Shift Effect

Sdelta reduced behavior prob & shifts discrim away from S^D

• towards a more extreme alternative stimulus

• resulting in a preference for stimuli that differ more from the original SD.

• ex→ trained to respond to specific tone & press bar, set expectations

  • X tone = nothing

  • lower tone = nothing, expectations now altered, even w/ a tone animal was already train then extinguished

  • new/higher tone added new context, seems lower in comparison

Multiple Schedules

can include 2+ schedules, X chained schedules because each response is reinforced

• org under stim control when they can complete each schedule

• animals adopt a typical response pattern for each schedule when they switch

• generating distinct behaviors depending on the schedule in effect

• ex → switch behaviors/showing up & focus w/ diff classes & professor expectations

Altering Schedules Example

Let us train two rats on this schedule, responding to colored lights:

• RedFR50 : Press → Food / GreenFR50 : Press → Food / RedFR50 : Press → …

• Rat A is switched to this schedule:

  • RedFR20 : Press → Food / GreenFR50 : Press → Food / RedFR20 : Press → …

• And rat B to this schedule:

  • RedFR100 : Press → Food / GreenFR50 : Press → Food / RedFR100 : Press → …

How might their behavior change?

• The behavior of each rat may change based on the reinforcement schedule adjustments. Rat A may increase its pressing rate due to a lower response requirement (FR20), while Rat B may show a decreased pressing rate due to a higher requirement (FR100). Each rat's response pattern will reflect the demands of their specific schedule.

Negative Contrast Effect

org's response rate to a less preferred stim dec following a shift from a more preferred stimulus, leading to a diminished expectation of reinforcement

• rat is on a superior schedule when red light (FR20) is on → richer/more dense

• response rate will inc for that schedule, but DEC for FR50

Positive Contrast Effect

occurs when an org experiences an inc response rate to a more preferred stimulus following a shift from a less preferred stimulus, enhancing the expectation of reinforcement

• red light is on (FR100), response rate will dec, but inc for green list (FR50)

Behavioral Contrast

refers to changes in an organism's response rate to stimuli based on previous reinforcement schedules, resulting in increased or decreased responding to one stimulus when contrasted with another.

• behavior changes even for a schedule that wasn’t altered

Anticipatory Contrast

org changes response rate when they know a change is coming

Errorless Discrimination Training

technique used to teach an org to distinguish b/w stim w/o the opportunity to make mistakes

• minimizes confusion & inc learning efficiency

• designed to minimize the number of responses made following S^delta

• minimizes frustration an org experiences

• achieved by fading a stim in/out slowly

Difference between Training a S^delta & Errorless Discrim Training

training extinction + downsides, errorless = no opportunity for mistakes, focusing on gradual stimulus fading

What can fading acheive?

reduce frustration, promote successful learning, create a smoother transition to new stimuli

• Terrace performed errorless training using fading to teach pigeons to discrim S^delta while making very few mistakes

• also used it to change a S^D & S^delta to new stim, in both, did it gradually

  • S^D - red light to white vertical line

  • S^delta - green light to white horiz line

Downsides to Errorless Training

may include slower overall learning, potential lack of flexibility when responding to new stimuli, and reduced opportunities for natural error correction

• ex → using it to train arithmetic helped a girl learn subtraction w/ fewer errors, used fc w/ answers showing & faded them out on repeat presentations until invisible

• ex → pigeons, made it harder to unlearn things, those undergoing training struggle to learn a new meaning to old stimuli (red = stop → red = go)

making errors may be an important part of learning complicated, flexible material

Targeting

training of an animal to engage w/ a specific item/location to complete a more complex task by guiding them with stimuli or rewards

• using stimulus control

• ex → dog might be trained to follow a stick, can be used to train gymnastic behavior

• ex → dogs might be trained to run to a laser mark, allowing search & rescue to direct them at a distance

• ex → sleep hygeine, go to bed when sleepy, X read/scroll in bed, X sleep in places other than bed, leave bed if X sleepy = help to all reinforce sleepiness w/ bed