NEcon Week 6 - Blocking and Dopamine, Algorithms, etc.

Waelti et al 2001 - Dopamine responses comply with basic assumptions of formal learning theory. explain graph

stage 1: associate “A” with a reward, the symbol A is predictive cue, the dashes are the monkey licking. monkey licks in anticipatiion between cue and reward and then licks after reward presented. 

graph on the right is data from recording dopamine neurons > recording from dopamine neurons after learning have no response to reward

Stage 1 asocaite A with Reward

A+ means its followed by reward after learning, dopmaine neurons respond to predictive cue (the stimulus symbol) itself, not the reward. 

Stage 2: present A and X then reward

Add another cue, X. but X is a blocked cue since the assocaition with A has already been learned. No activity after the Reward

Stage 2: present A and X then reward

since learning already occured, dope neurons respond to the stimulus, neurons fire for both cues

Test phase X. Does the Monkey learn that X predicts reward?

monkey does not lick after presentationof X (no assication performed despite X consistently paired with reward. the blocked cue, X, does not elicit reward prediction error.

X has been blocked by A from entering an assocaition pairing with the reward so after X the monkey has no expectation of reward

Test phase X. Does the Monkey respond to X cue?

no reward prediction error = no learning, no response of dope neurons to blocked cue X either

what happens if you unexpectedly deliver reward after X cue during extinction?

the X cue is meaningless in prediction, so the reward is unexpected and dope neurons respond to reward > dopamine neurons signal violations of expectations at time of reward for assocaition to take place

In the second half of paper Waelti et al 2001 start new experiment. Example#2

present stimulus B alone by itself with no reward, the dope neurons are not active at all

What happens when you add new cue Y along with B and start presending a reward? Early stage 2

after repating that pairing, monkey starts anticipating licking for BY stimulus

dope neurons respond in reward prediction error during learning because early in training neither stimuli predict reward is coming, an assocaition between cue and reward should be made

response to BY stimullus after learning? Stage 2

dopamine neurons between responding to BY stimulus once learning has occured

what happens when Y presented alone with no reward followed (extinction)?

violation of expectation - monkey is licking after Y presented ,  dopamine neurons respond to the stimulus Y suggesting assocation between Y and reward has occured

dopamine neurons stop activity when no reward presented during usual reward time 

what if reward delivered early in extinction with Y is presented alone

no change in dope response to reward since the reward is expected,  dope neurons responds to the predictive Y cue

true or false, dopamine signalling conforms to learning theory, reward prediction error encoding is correlated with associative learning

impact of negative vs positive coorelation

true

positive reward prediction error strenghten association (dope neurons dont respond to predicted reward, respond only to predictive cue.)

negative reward prediction error weakens assocaition (dope neurons stop activity)

Steinberg et al. 2013 - a causal link between preduction errors, dopamine neurons, and learning.

what was goal of paper

what is optogenetics

goal is to prove error signaling by dopamine neurons causes assocaition learning

optogenetics > using light to selectively activate specific cell types expressing ligtht sensitive proteins called opsins. cause AP to occur at precise times and durations (turn on and off neuron at ur leisure)

what is ChR2-eYFP

channelrhodopsin protein light gated cation channel > stimulation with blue light activated channel and depolarizes neuron, can control timing of spikes 

Steinberg et al. 2013 - a causal link between preduction errors, dopamine neurons, and learning.

Blocking task and pattern of delivery

Blocking Task: sound cue > water drop, sound cue/light (compound cue) > water, light > water

Control Task: water, sound cue/light (compound cue) > water, light > water

the cue is delivered for 30 seconds and signals availability for the water reward, the rat has to put its nose into a port to receive the reward, once in the port reward is delivered 6 times in 3 seconds intervals as long as rat remains in port

what does this graph indicate?

a measure of how much time the rat stays in the reward port, as the assocaition between the single cue and reward gets stronger rat stays in port longer. no difference between blocking and control group

and this graph?

time spent in port during compound cue stage

blocking compound cue = original tone + light combo > water

control compound cue = same tone as blocking + light combo > water

rats spend same amount of time in port, shows association forming in compound cue stage

critical test stage results

control group spend significantly more time in the port than the blocking group > so the blocking worked. 

explain optogenetic experiment by Steinberg et al. 2013 - a causal link between preduction errors, dopamine neurons, and learning.

three groups:

2 experiment groups prepared for opto,

1 control group, dont have protein expressed for opto to work

during the compound cue stage, the experimental group 1 are going to have dopamine neurons excited while reward is delivered, other experimental group 2 get dope neurons excited after reward

what results of compound cue phase show?

rats in all groups stayed at the well same amount of time during single and compound cue phase

what were results of the final test phase (presenting only light)?

during compound cue phase, dope neurons dont respond at time of reward

during paired stimulation, an artificial positive reward prediction error signal is given, we see animals in this group spend more time at the port compared to other groups. stimulation during reward unblocks the blocked cue

unpaired stimulation does not support learning

• Dopamine stimulation DURING reward delivery

  • When the rat gets the reward, the dopamine neurons fire because of the light.

  • This adds a “fake” reward prediction error — it makes the brain think,

    “Wow! The reward was better than expected!”

  • Because of this extra dopamine burst, the brain now learns that cue B (the light which was technically the blocked cue) predicts reward, even though the real reward wasn’t surprising because it was not expected in the first place.
    ✅ Learning about cue B happens.

• Dopamine stimulation AFTER the reward

  • If you activate dopamine neurons after the reward, it’s too late — the brain has already updated its expectations.

  • The dopamine signal no longer aligns with the unexpectedness of the reward.

  • So it doesn’t act as a teaching signal.
    ❌ No learning about cue B occurs.

describe the extinction experiment Steinberg et al. 2013

3 groups of rats go through training phase to associate cue with ducrose reward, during extinction phase (withoold reward) paired group gets stimualted after reward woud have been delivered, continued stimulation sustains learned behaivor compared to controls, indicating the burse capacity to substitues for the omitted reward signal.