Week 10 Notes: What is actually learned during Learning?
stimulus Reponse Theory: Pavlovian Conditiong
when we present a CS there will be a cascade in the neural activity when learning is happening
before learning happens the CS will not evoke any conditioned response and only the US will evoke an unconditioned reponse
we expect no response
When we present a US, the US will evoke a cascade of neural/psychological activity
After the cascade we expect the ripple effect to leads to a UR
Both events cause changes in neural activity
We might be creating a link when learning is happening to result in a CR
After learning an association the CS will lead to a CR
perhaps learning will establish a new brain link between the CS and the UR
Stimulus-Response (S-R) Pavlovian Learning: During learning, a new neural pathway is created which between a stimulus (CS) and the neurons which control for the unconditioned response
S-R Theory Prediction:
CR should always be the same as the UR (since the links of the CR will leads to the pathway of the UR) * we know that this isn’t true
Food US: UR→ salivation CR → saliviation
SHOCK US: UR → jump CR→ freezing
No learning should ever occur without a UR
we know that this is also false due to examples such as sensory pre-conditioning where learning is still happening even though the animal is exhibiting no conditioned response
There are instance where the S-R association can exist!
Stimulus-Stimulus Pavlovian Learning
Definition: During learning, a new neurla pathway is formed between neurons directly activated by a stimulus (CS) and neurons normally activated by the event that follows a US. Thought of as being S-O (stimulus - outcome) association.
When the dog hears the bell then it will lead the subject to think about food
the bell doesn’t directly activate the salivary glands
Some associations are formed by S-S learning and S-R learning
S-R and S-S Learning
S-S Learning: mentally evoke the representation food which will lead to the activation of the salivary glands
The bell will leads to subject to think about the US which leads to the UR
S-R Learning: bell representation in the brain will lead to the activation of the salivary glands due to the connection of the neural pathways
The bell itself leads to the salivation (UR)
Direct short circuit
Associative Content of S-S and S-R Pavlovian Learning
S-R Association: Association is stored to act as an instruction, association is less flexible
Will not know why the response is taking place
No real thinking is require here
Faster to lead to a change in behavior
S-S Association: association conveys knowledge about what is going to happen next
Subjects learns that the CS is a predictor
More flexible to use
Provides knowledge of what is happening in the world
Contains more informations of what the actual outcome is going to be
More thinking and processing is going to happen
US Revaluation as an Index of What is Learned
q: how can we tell the associations apart?
US Revaluation Procedure: Post-learning changes to some property of the US ared used to probe the content of associations by testing for changes in conditioned responding
Animals could be showing the same type of response but the causes can be different for each of them
We can devalue the US (devaluation)
We can increase the value of the US as well
Test: we would give animals food → bad taste
If an animal shows a S-R response
We would expect the level of responding to remain the same since the US is activating the salivary glands themselves
Food is not activating the salivary glands
No change in the level of responding
If an animal shows S-S response
knows that tone → food and food→ illness
We would expect a decreasing in responding
CS is causing the animals to think about food and the response itself was caused by the mental representation of the US
There would be a change in responding
Instrumental : lever press → food
tone → shock pairings are given
Revaluation occurs : one group is habituated to the shock (can ignore) and another group gets no training to the shock
Test: measuring conditioned suppression
S-S: shock doesn’t both the group, no fearful response
S-R: hearing the tone will directly activate the fear response, the fear response should be happening anyway
we are changing the value of the US during both of these examples
Rats will get the same learning and some rats will learn S-R or S-S evaluations
We usually can’t guess what type of association is going to be learned
tools let us understand what type of associations are going to be learned
What is Learned during Instrumental Conditioning?
Stimulus → Response
Responding is independent of the identity of the reinforcer
Law of Effect
Habitual Behavior → environment will elicit the behavior
No sense in which the outcome is what is motivating the behavior
Consistent with Thorndike
Response → Outcome
Responding is based on beliefs and desires
Cognitive Theories of Learning
Goal directed behavior
Are what we typically think of when we think of instrumental learning
Subjects are aware of the potential outcomes and will be responding in way where they are purposely working toward the reinforcer
Goal-Directed Behavior (subjects are working towards a specific outcome)
Drive more complicated behaviors
Both types of associate are difficult to dissociated when seeing a subject act a certain way
Procedures used to identify instrumental associations
Plus-Maze Task
Reinforcer Devaluation
CTA
Sensory-Specific Satiety
Contingency Degradation
Pavlovian- Instrumental Transfer Effects
Dynamics of Goal-Directed & Habitual Behavior: Plus Maze Task
Procedure: The rat will be placed in a maze where it will have the option to turn left or right and will only be awards with a food pellet if it turns left
Rat needs to learn that the location is associated with food
Q: What type of association is learned? S-O or S-R association? How can we test this?
We are able to do probe training, unexpectedly start the rat on the opposite end of the maze
Interpretation:
If the rat goes to the place where food was previously delivered then it will show a S-O association as it will associate the location with the reward
If the rat just turns left then the association is going to be S-R as it will automatically turn left
Early in training, behavior is goal directed (R-O system)
After extensive training behavior transitions to habitual control (S-R system)
Makes sense because with the more practice you have the more automatic a behavior becomes
The timing during which a probe test is conduct will can affect which type of association is being shown by the subject as the association can change over time
Animals will frequently learn both S-R and R-O associations
after responding becomes habitual, if we lesion the dorsal striatum the place strategy (R-O) will re-emerge
Dorsal striatum is critical for showing habitual learning
Shows that the previous R-O strategy still exists in the brain and is used by the subject when necessary
If we lesion the hippocampus without any training have happened then the subject will only learn habitual strategy
There is competition between brain regions for the control of behavior in which there's a transfer between goal-direct to habitual learning processes
Both Goal-Directed and Habitual Behaviors are useful
When there’s place strategy (goal-directed) the subject will be able to have more flexible behavior
ex. being able to readjust your way to work when there’s a roadblock
When the subject shows more of the habitual behavior, it allows for faster and more efficient behavior that is rigid and less flexible
ex. automatically driving your usual route to work
The transition between goal directed behavior to habitual behavior shows the best of both worlds
Good to have both of the association to lean on
Distinguishing Goal-Directed & Habitual Behavior: Reinforcer Devaluation
Three main steps
Training
Devlauation
Test
1st: Training
Teaching two pavlovian associations
Two actions lead to two outcomes (two different types of food pellets)
One lever leads to grape food
one lever leads to cherry food
Devaluation Test:
Sensory Specific Satiety: feed rat a lottt of food pellets (have the animal full on one of the food pellets)
Conditioned Taste Aversion: have the grape pellets be associated with illness
Test:
Look at how frequently the rat will show both of the trained outcomes
Habitual: we seen that the frequency at which a rat presses both of the lever stays the same
Just seeing the lever leads to the rat to press them
Goal Directed: we see that the rat will decrease the frequency of the food outcome that was devalued while the the non-devalued lever will have a higher lever pressing rate
Shows that the subject will work to earn the outcome it desires
The amount of instrumental training will affect whether behavior is under habitual or goal-directed control
We can see that when we conduct this type of test when the subject has more experience it will show more results of equal amount of responding to both types of reinforcers since it will show more amounts of habitual learning behaviors
Distinguishing Goal-Directed & Habitual Behavior: Contingency Degradation
If the goal is goal directed
Same type of training procedure as the reinforcer devaluation
A reinforcer will only be delievered when the subject presses a specific type of lever
Contingency degradation:
For one of the reinforcer: The frequency of getting the reinforcer is the same whether or not the rat will press or not press the lever
Pellet is delivered during their lever press breaks
Test:
The rat will have the option to press a lever
Goal-Directed: We expect that the rat will dectrease their lever presses for the reinforcer that is always being delievered since it should know that the food is going to be delivered anyway
Habitual: we would see the rat continue to press both of the levers regardless of the contingency degradation
We see similar results for
test of habitual control
Pre-exposure to drugs
Pavlovian devaluation
Suggests long-term exposure to drug of abuse strengthen habitual control of behavior
contingency degradation is the instrumental version of truly random control
changing how tightly a subject’s actions are equal to the probability of the subject getting the outcome
Pavlovian-Instrumental Transfer
Wtn to see if there are any moments where a pavlovain association will change the way an instrumental baehvior si carrried out
ex. Pavlovian association (with an aversive stimulus, shock) will terminate the instrumental behavioral outcomes
A CS associated with an aversive outcome causes a decrease in instrumental behavior
Pavlovian-to-Instrumental Transfer (PIT): A pavlovian CS can increase or decrease ongoing instrumental behavior
A CS+ increased instrumental behavior
A CS- will not affect instrumental behavior and can at time decrease instrumental behavior
Pavlovian Instrumental Transfer success will shows that Pavlovian associations and instrumental associations were not S-R
we wouldn’t expect an increase/change in the behavior if the associations were S-R since it wouldn’t cause the subject to think about the reinforcer when the pavlovian association is shown
General Pit: a pavlovan association between a CS and the motivation properties of the US ( appetitive/aversive) energizes or de-energizes instrumental responding
only established one pavlovian association so we wouldn’t be able to determine specificity to instrumental learning
Specific PIT: A pavlovain association between a CS and specific representation of the US eneegizes instrumental responding reinforced by the same US
ex. training three different associations (three separate pavlovian associations), two instrumental learning responses
We pair both of the pavlovian associations with each of the instrumental learning response in which we will see that each of the pavlovian associations will have specific effects on the response rate of the lever presses for a specific reinforcers
There is selective enhancement of the specific instrumental learning
We have the largest amount of evidence that the Pavlovian association is S-S and the instrumental learning association is R-O
For the general behaviors we see that the change to the instrumental responding is not specific and the pavlovain association will cause the same amount of change
Specifc PIT depend on an R-O (goal-directed) instrumental association and an S-S pavlovian association
General PIT: A pavlovian association between a CS and the motivational properties of the US energiezes instrumental responding
good leads to good
easy to obtain these types of results
animals aren’t purely using S-R association
points in the direction of not using S-R
Specific PIT: Pavlovian association between a CS and a specific representation of the US energizes instrumental responding reinforced by the same US
more energy is allocated a specific response
PIT provides a useful tool for analyzing the content of instrumental and Pavlovian associaiton