yas- CM2

Lecture 2: Blocking and Extinction (beyond prediction error!!!)

Contact Information

• Email: Christopher.mitchell@plymouth.ac.uk

• Office: PSQ B229

• Office Hours: Wednesdays 2-3pm, Fridays 2-3pm (subject to change)

• Check-in code: Prof Chris Mitchell

Overview

• First Half: Blocking

  • Reminder of blocking

  • Prediction error: Rescorla Wagner

  • An alternative ‘attentional’ account of blocking

• Second Half: Extinction

  • Protection from extinction

  • Context specificity of extinction

• Reading: Pearce (2008) Animal Learning and Cognition, Chs 3 and 5

Experiment: Conditioned Response to Colored Squares

• Procedure:

  • Participants are presented with one or two colored squares.

  • Sometimes the presentation is followed by an electric shock.

  • Task: determine which colors lead to shock.

• Shock Intensity: Double shock if both colors present lead to shock individually.

Standard Blocking Result

• Presentation of colors (e.g., Brown, White, Orange, Purple) followed by shock.

• Blocking Occurs: Prior training with one stimulus (A) blocks learning about a second stimulus (B) when they are presented together (AB).

  • Example: Brown > Purple

Associative Links

• Blocking is usually explained as a failure to form a link between cue B and the US.

  • Example: Orange -> Shock!, Purple (no link formed during blocking).

Prediction Error in Blocking

• Design: A+ then AB+ ('+' indicates presence of the US, A & B are Conditioned Stimuli)

• By the end of A+ trials, the US is fully predicted by A; therefore, there is no prediction error.

• During AB+ trials, the US is not surprising because A already predicts it, so B isn't processed.

• No B-US association is learned because AB+ trials are the only time B appears.

• Learning about B is 'blocked' by A.

Importance of Blocking

• Before blocking, it was thought that associations form simply by pairing two stimuli (Hebbian learning).

• In blocking, Cue B -> US on AB+ trials, but little learning occurs.

• Demonstrates that learning isn't just about pairing stimuli; prediction error matters.

• Central to contemporary learning theory.

Main Point: Prediction Error

• No prediction error on AB+ trials after A+ trials because the US (+) is not surprising.

US Potency

• The US becomes unsurprising and less potent over repeated A+ trials.

• Comparison: Blocking (A+ then AB+) vs. Overshadowing (CD+).

  • In overshadowing, the US is surprising on initial CD+ trials.

Formalizing Prediction Error

• Surprise: The difference between what you predict will happen and what actually happens.

Rescorla-Wagner Model (1972)

• \Delta V = \alpha(\lambda - \Sigma V)

  • \Delta V = learning: change in associative strength on this trial (blocked cue B on AB+ trials)

  • \lambda = total associative strength supported by the outcome (US)

  • \Sigma V = the expected outcome (given all cues present A & B)

  • \alpha = attention to cues (salience of the CS); this doesn’t change with learning according to R&W

• On AB+ trials, \lambda = \Sigma V because of the pretrained 'blocking' cue A. Thus, no learning occurs for B.

Main Point of Rescorla-Wagner Model

• Learning occurs only when there is prediction error (i.e., when you are surprised by the outcome).

Alternative to R&W: Attention to CSs

• Signals of important events are attended to especially well.

• The salience of the CS “\alpha” is not fixed (contrary to R&W’s claim); it can change with experience/learning.

• Blocking might occur because attention to B decreases due to the presence of the better predictor A.

Example of Blocking: Plymouth Argyle

• Plymouth Argyle loses every week.

• Buy Bellingham -> win every time he plays.

• Buy another player (Palmer) -> win again.

• How good is Palmer? Evaluated under conditions of blocking.

Blocking and Divided Attention

• Will Plymouth win? Look for Bellingham (attend to him).

• Palmer processed under divided attention.

• At test, Bellingham's contribution is clear, but Palmer's is less so due to blocked learning.

Mackintosh (1975): Blocking Due to Changes in \alpha

• A+ then AB+

• The most predictive cues will increase in salience/\alpha (e.g., cue A).

• Relatively less predictive cues will decrease in \alpha (e.g., cue B on AB+ trials).

• Not much learning about cue B.

Measuring Attention: Eyetracking

Eye Gaze in a Learning Task (Beesley and Le Pelley, 2011)

• No shocks.

• Present stimuli on the eyetracker screen.

• Cues: chemicals (Addexium, Rezaline).

• Outcomes: symptoms in Mr. X (itchiness, nausea).

• Do participants look at the word 'Addexium' (the CS)?

Experimental Design

• Stages

  • Stage 1: A – o1; C – o1; E – o2; G – o2; I – o1; L – o1; O – o2; R – o2

  • Stage 2: AB – o1; CD – o1; EF – o2; GH – o2; JK – o1; MN – o1; PQ – o2; ST – o2

  • Stage 3: BK – o3; DN – o4; FQ – o3; HT – o4; BN?; DK?; FT?; HQ?

  • Test: BT?; DQ?; FN?; HK?

Results: Eye Gaze in Stage 2 AB+/JK+ Trials

• Attention to cue A is high (good predictor of the outcome).

• Attention (eye gaze) to 'blocked cue' B is reduced compared to overshadowing controls J and K.

• Less attention/processing of Cue B would explain why there is less learning about that cue on AB-O1 trials.

Blocking Summary and Conclusions

• Blocking is seen in humans (e.g., Beesley & Le Pelley, 2011) and in non-humans (e.g., Kamin, 1969).

• Two possible mechanisms for blocking:

  • Prediction error: No learning because US isn’t surprising on AB+ trials.

  • Reduced attention to cue B on AB+ trials due to higher salience/better predictor cue A.

• Evidence for the salience mechanism in humans (Beesley & Le Pelley, 2011).

• Blocking is not all about prediction error.

Part 2: Extinction

Why is Extinction Important?

• Some aspects of extinction don’t fit with Rescorla Wagner.

• Clinical relevance: phobias and PTSD.

Extinction Experiment: Thug Recruitment

• Assessing which candidates (Zack, Jeb, Paul) are evil enough to join a gang.

Prediction Error in Extinction

• Early in training: All evil deeds are a surprise.

• As you hear more about them, you begin to expect evil from Jeb and Paul.

• Later in training: Expect evil from Paul but experience no evil: Negative prediction error.

• Association between Paul and Evil reduces later in training?

Extinction: The Design

• Phase 1: Paul-evil (A+)

  • Come to expect the US

• Phase 2: Paul-no evil (A-)

  • Surprising absence of the US -> Prediction error on first trials

Extinction Curves

• Initial learning curve: US becomes less surprising over trials; increase in associative strength slows down.

• Extinction curve: absence of US becomes less surprising over trials; decrease in associative strength slows down.

Rescorla-Wagner and Extinction

Renewal

• The CR returns in a different context from extinction.

Spontaneous Recovery

• The CR returns with a delay between extinction and test.

Example of Renewal

• Same task: which ones are evil enough to be let in?

Renewal: Extinction is Context Specific

• Learnt that Paul does evil stuff in Plymouth.

• Extinguished in Lamonic Bibber.

• Plymouth and Lamonic Bibber are different 'contexts'.

• Paul -> Evil association returns when back in Plymouth.

A Model of Renewal (Bouton)

• Paul -> Evil, Lamonic Bibber

• Reminder: Plymouth Paul-Evil

• Lamonic Bibber Paul-Neutral

• Plymouth Paul? -> New learning

Renewal Language

• Context A: CS-US

• Context B: CS-

• Test: Context A CS? CS US

• Context B: New learning

Most Important Thing

• Extinction is specific to the context in which it was learnt.

Bouton on Extinction

• Extinction is new learning of an inhibitory link.

• That inhibitory link only operates in the context in which it was learnt.

• Rescorla Wagner is wrong: The original CS-US link does not weaken.

Exposure Therapy and Extinction

• Exposure therapy seems to be the same as extinction.

• Both show renewal and spontaneous recovery.

• How to avoid context specificity of exposure therapy (Laborda et al, 2011):

  • Have therapy in lots of different contexts.

  • Spread exposure treatments over a long period of time.

Important Things

• Prediction error (e.g., Rescorla Wagner model) explains:

  • Learning curves (A+)

  • Conditioned inhibition (A+/AX-)

  • Blocking (A+ then AB+)

  • Extinction (A+ then A-)

• Need additional explanations for:

  • Second-order conditioning (within-compound associations)

  • Attentional effects in blocking (e.g., Mackintosh, 1975)

  • Context specificity of extinction (inhibitory link that is activated by context)

Readings

• Pearce (2008) Animal Learning and Cognition: An introduction, Chapter 3 (particularly pages 64-71; 83-86) and Chapter 5 (particular pages: 125-128; 134-138)

• Additional: Beesley, T., & Le Pelley, M. (2011). The influence of blocking on overt attention and associability in human learning. Journal of Experimental Psychology: Animal Behavior Processes, 37(1), 114. Laborda, M. A., McConnell, B. L., & Miller, R. R. (2011). Behavioral techniques to reduce relapse after exposure therapy. Associative learning and conditioning theory, 1, 79-104.

Spontaneous Recovery

Bouton’s Model of Spontaneous Recovery

• Time A CS-US

• Time B CS-

• Time C CS? CS US

• Time A New learning

Bouton and Spontaneous Recovery

• CS-US conditioning at Time/context A

• Extinction at Time/context B

• Test is later at Time/context C

• Initial conditioning transfers from Time A to Time C

• Extinction does not transfer from Time B to Time C

• Spontaneous recovery is a form of renewal (ABC renewal).