Animal Learning - PSYC1011

Lecture 1
What is learning?
Learning is the acquisition of knowledge or skills through experience
Typically, learning is revealed by a change in behaviour and the brain; these changes allow an animal to adapt to their environment
Animals that are better learners might gain advantages
Obtaining food
Reproduction
Avoiding predation
Learning is important for dealing with these challenges
Sometimes, however, an animal fails to learn or they might even learn the “wrong” thing (both of which can lead to problems)
Mental health is the number ONE health problem in developed countries, in terms of years lost to disability (costs more than all cancers combined)
US National Institute of Mental Health committed US $1.54 billion in 2017 in funding to reduce “the burden of mental illness and behavioural disorders through research on the mind, brain and behaviour”
Learning is involved in some of these disorders
Anxiety, and related, disorders (e.g., phobias, PTSD)
Also, individuals with mental health problems may learn about the world in a different way
Could affect how they interact with others, and how they respond to treatments
Why do we study “animal” learning in psychology?
We are animals.
All animals have particular, similar challenges
Maybe other animals solve challenges the same way we do
Continuity of species - Darwin
Focused on anatomical similarity (e.g. wings)
Both species have to solve certain problems (e.g. bat vs pigeon), very different species but both have wings to solve the same problem
Very very high degrees of similarity at some stage of development (e.g. embryos look so similar yet develop extremely differently)
Similarities that we see across species, e.g. how we respond to predators
Also in the brain, in neural processes that are very similar
There is a molecular cascade of memories that happens in snails, monkeys, rats, to our knowledge even in humans
All animals when they form memories experience this memory cascade
The amygdala is important for emotional learning across several species
Might the same principle (continuity of species) apply to learning?
Helps animal adapt
Instantiated in the brain
Lots of potential applications
Can help us understand:
Our likes and dislikes (maybe many of those are learned early in development?)
Anxiety disorders, and their treatment
Rodents especially help us learn about how anxiety disorders are eased (CBT)
Neural bases of learning and memory (drugs to improve memory?)
Non-associative learning
A type of learning where an animal’s behaviour/physiology/brain changes following repeated exposures to a stimulus
Two types
Habituation - a decrease in response amplitude or frequency as a consequence of repeated experience with a stimulus
Is perhaps the most common type of learning
Is stimulus specific
Critically important for functioning in the world
Allows the animal to avoid the familiar, focus on the new. We have limited attentional capacity, we are bombarded by sensory information. We need to ignore the familiar
Heart rate orienting response
Adult rats walking around, they hear a beep beep sound, first time they hear it, their heart rate goes down, but the more they hear it they respond less. Even if there is a time break, they will remember the sound and know not to respond
Infants are not the same, they forget much more rapidly (infantile amnesia)
Mutschler et al (2010)
Expose pps to a classical piano piece for 40 mins, they ask them how arousing do you find this, how much do you like it, while pp is in an fMRI. Their initial liking response and arousal gets lower and lower as time goes on - they get bored.
Amygdala response gets lower too
Gandhi et al (2021), measured skin conductance responses to a repeatedly presented sound (a metronome) in a group of young adults that had been diagnosed with autism spectrum disorder (ASD) or not (labelled as neurotypical)
ASD: Same magnitude initially as NT pps, but as time goes on, they don’t habituate, they keep responding to it
They are learning about the world in a different way
NT: They are responding 20% less than they were at the start
Also applies to other types of stimuli e.g. in our species faces convey a lot of information, and we respond to a new face
The neural response (in hippocampus) to a fearful faces habituates in non-diagnosed adults (Breiter et al., 1996)
Big difference between NT and those with Schizophrenia, more activity in the latter’s brain. Non diagnosed people habituate. Hippocampus continues to pay attention to this person.
Very simple type of learning can tell us about how all animals react to their ever-changing world. Further, it may give us insights into how individuals with a particular psychopathology react differently to the world
Sensitisation - an increase in response amplitude or frequency as a consequence of repeated experience with a stimulus
Lecture 2
Habituation (continued)
Benito et al (2018)
Benito et al (2018) examined habituation and treatment outcomes in a group of individuals diagnosed with pediatric OCD
Looked at 100+ of patients who had received CBT for OCD
Found single best predictor of treatment success in all aspects of treatment success (global improvement, reduced symptoms etc), was habituation of fear response
Habituation increases, and OCD symptoms change
Did not mattered how much you were exposed to it, but only that they were habituated to that fear
Benito et al. : the most important factor to consider when thinking about whether treatment is likely to be successful is “the degree to which exposures provide a relevant and/or potent learning experience”
Holz et al (2021)
Measured the neural response to emotional faces (angry/fear) in the amygdala
Participants were on average 25 years old, but had been apart of this study for entire life
Parents signed up at birth (longitudinal study)
At 3 months old, came into lab with mother, video-taped during a lab visit, and the amount of maternal interaction (i.e., “maternal stimulation”) was determined
Classified maternal interactions as high (playing with kid, lots of love) or low maternal simulation (not as much attention to the child e.g. being on their phone)
Results:
Lifetime diagnosis of ADHD was weakly associated with habituation of the amygdala response (i.e., less habituation in those diagnosed with ADHD)
The amount of maternal stimulation affected habituation but ONLY in high risk families
Only did so in those from “high risk families”
means that one or both of their parents had been diagnosed with a psychiatric condition, then high risk
Level of maternal engagement can protect one from coming from a high risk family
Remember, this measure of maternal stimulation was obtained when the 25 year old participants were only 3 months of age
This indicates we can get a really early measure of who is high risk and even HIGHER risk
Sensitisation
An increase in response amplitude or frequency as a consequence of experience with a stimulus
Sensitisation is more likely through strong stimuli, like electric shock
Is not stimulus specific
Can memory be “transferred” from one animal to another (even though the “other” animal didn’t have the relevant experience)?
E.g. you get angry hearing a song you don’t like, and then get angrier and angrier every time they hear it
Bedecarrats et al, 2018
A group of adult Alysia → 5 mild electric shocks to their tail
24 hours later, they get 5 more
They are tested 24 hours, test is a water squirt to their siphon, how long does the siphon withdrawal for
Other group is tested the exact same but without the shocks
Result: Animals shocked stayed withdrawal 10s longer
They took RNA from control and trained group
Gave the RNA via injection to untrained animals
Animals given trained RNA, had the near same reaction to those who experienced it, despite never having experienced the shocks

Boileau et al. (2006)
Measured brain response to amphetamine (dopamine release in striatum)
3 doses of amphetamines via injections in healthy adults
2 weeks later, second injection
Then 1 year later
Brain is way more fired up after each dose
Way more dopamine release after each time
Summary
In habituation/sensitisation, animals learn to recognise that event is familiar. But, they do NOT learn anything about the relation between that event and any other.
Learned relationships between events linked in time and/or space are called associative relationships
Pavlovian conditioning (associative learning)
Animals learns that one stimulus predicts another stimulus (i.e., that goes with that)
Originally studied digestion
Unconditioned stimulus
Food in example, usually a biologically relevant stimulus
Unconditioned response
Salivation, unlearned response in US
Conditioned stimulus
Bell in example, a neural stimulus that does not initially elicit the UR
Conditioned response
Salivation in example, response to the CS after learning has occurred

Important for our emotional reactions, as well as many other surprising things
Lecture 3
Associative Learning
Experimental “neuroses”
Pavlov’s dog study
Phase 1: the dog is brought down to lab and put into harness, periodically a circle is presented for 10 seconds, and at the end of the 10 seconds, it gets food, this is repeated. It should end up salivating when the circle is presented.
Phase 2 (discrimination): dog is in a harness sees the circle and gets food, but it is also going to see an oval shape, when presented oval he does not get food. Dog still salivates at the oval. Overtime, the dog will learn to discriminate.
Final phase: They see the circle and get food, but when they see another shape that looks exactly like the circle, they don’t get the food.
This totally changed their emotional state, dog originally is placid and calm. It gets angry and neurotic.
Pavlovian conditioning also very important for emotional reactions in humans
Simple illustration of this is a study by Olsson & Phelps (2007)
One stimulus (e.g. blue square on computer screen) predicted shock while a different stimulus (e.g., red square) predicted no shock. These two stimuli are referred to as the CS+ and the CS- respectively
Measured Electrodermal Activity (EDA AKA skin conductance) in skin to the new stimuli
CS+ starts to elicit a state of fear (skin conductance response increases), pps showed more reaction to CS+, they also measured neural activity in the amygdala (MRI)
Amygdala had much more activity when presented with the CS+
We can learn about the associations between two stimuli even if we don’t experience them
The highly social word that humans live in provides many opportunities to observe others’ emotional reactions to various stimuli
Observational (vicarious) learning - learning that occurs from simply watching another (i.e., no direct experiences of US)
Olsen & Phelps (2007) → extended experiment, participants watched other participants seeing the blue and red square
Measured EDA, pps in the two conditions exhibit comparable learning!
Also measured neural responses to the two stimuli (focused on the amygdala), showed exact same response despite never having experienced it
Pavlovian conditioning thought to be especially important for the learning of emotional reactions (e.g., what we like)
Long-standing view in psychology is that early experience have an especially pronounced impact
Brunjes & Alberts’ study of huddling behaviour
circular chamber: one side has an anesthetised adult gerbil and on the other side is an anesthetised adult rat (they are laying down not moving because anesthetised), they dropped 10 day old rats and 15 day old rats into the circular chamber to observe
10 day old rats 50% with rat and the gerbil
15 days old, 80% with rat, 20% gerbil
This is referred to as a transition from physiological to filial huddling
At first, they just want a warm body so they want to stay warm with whoever, but then 5 days later, they want to stay warm but also want to do it with rats
Alberts and May asked whether this transition was due to “nature” or was it due to “experience”
Rats suck their mother’s nipple and sometimes milk comes out
US = food, CS = smell
Nursing mothers were sprayed with either “wild musk” or saline (odourless/ normal smelling)
Rats tested from 5, 10, 15 and 20 days old
5-10 days don’t care, just want warm body (50-50)
15-20 days the change occurs, if rats were raised by a rat that smelled like wild musk, then they spend 80% of their time with a rat that smells like wild musk. If reared by normal smelling rat, they spend 80-90% with normal smelling rat
Suggests the transition from physiological to filial huddling is because of Pavlovian conditioning, they are conditioned by smell
Fillion & Blass (1986), rats were reared by a mum that smelled “normal” or of “citral” (odour sprayed on her ventrum each day)
Pups were separated from mum at 21 days of age, and male offspring tested when they were 100 days off age
Test consisted of being placed in a chamber with a female that was in heat
Prior to test, some females sprayed with citral, others with saline
They tested the time it took for the rats to ejaculate, depending on the smell of the female
When rats ejaculate quicker when the rat smells like their mother rat
Olfactory learning reported by Sullivan
In a typical experiment, she pairs an odour CS with mild electric shock US in rats of various ages (has two control groups at each age). She then measures odour preference (5 test trials)
Ages 6, 12 and 20
There are typically two control groups
1: Odour, no shock
2: Odour 5 times and shock 5 times but it is random, not predictable
At older ages, they avoid the odour when paired with shock
6 day old, they love the electric shock? They keep going for the CS odour
Explains child abuse, they are still attached to the abuser and love them, potentially this learning at a young age about a primary caregiver - even if there are bad things associated with them
Tiffany Field: Massage therapy in premature human infants
She did an experiment to massage the baby 15 mins a day
First experiment, just nurses
Grew fasters, discharged from hospital 6 days sooner; average hospital savings of $10k per infant = 4.7 billion dollars per year in US alone (Field, 2004)
Second, mothers
Third, depressed mothers
Allowing the depressed mother to rub the baby helps the post-partum depression symptoms
Over 30% higher survival rate for babies rubbed than non-rubbed
Lecture 4
Olfactory Learning - Sullivan (continued)
Learn that odour predicts shock
The youngest age group learn a preference for that odour, which is very puzzling
The findings change if you train the rat in the presence of their mother
Mother is not getting shocked, but just present
12 day old will now prefer the odour that predicts a shock
Tottenham et al (2019)
Tested 3-5 year old children
They were first trained that one shape (blue square) sometimes predicted an aversive, loud scream; this stimulus is referred to as the CS+
A different shape (purple triangle) never predicted the aversive US; this stimulus referred to as CS-
Some trained alone and others in presence of parent
After training they are tested
In a room behind a door there is a playroom, child is told that there will be a playhouse inside
There are two arms in the room

They are told they can go into either arm of the room, they are told both has a prize
3-5 year old child sees a certain geometric shape, and hear a scream
Parents have left at this point
When kids were not trained by their parents, they preferred the CS-
55% of time, kids trained with mother prefer the CS+
Pavlovian conditioning and Criminality
Gao et al., 2010
Trained ~900 kids (3 yrs old) across 2 years on a fear conditioning task (i.e., CS+ = shock, CS- = no shock)
Was done in Mauritius
Then waited for 20 years for the data of interest!
137 of those kids were convicted of serious crimes (rape, assault, murder etc)
750 were not convicted
They looked at the demographics of all the kids
Tried to match non convicted and convicted kids, this was control group
200 kids that were not convicted but had similar demographic to convicted kids
The children who grew up to be convicted of serious crimes, they did not exhibit the learning, the fear response to the CS+ - no increase in skin conductance
Birbaumer et al (2005)
Fear conditioning in psychopaths
Devoid of emotional reactions
All male pps, all incarcerated individuals, all found guilty of serious crimes (murder, repeated rape etc)
Psychiatrists had diagnosed some as psychopaths
Some were not psychopaths (control)
CS+ and CS- = neutral faces
US = painful pressure
Measured subjects’ ratings of CS-US contingencies, SCR to CS+ and CS-, and neural activity to each CS
Contingency ratings → how likely it is that I will experience pain when certain face shown, no difference between both groups, they both learned
Skin conductance responses extremely different, increase in SCR for control group for the CS+
The psychopaths do not respond to it at all, they do not experience the emotional reaction
They also measured neural activities, in healthy controls, there are parts of the brain that are much more active when seeing the CS+, but the activity is not there for psychopaths (less activation in the amygdala and the orbitofrontal cortex)
The psychopaths still experience the same pain, but they don’t learn!
Orbital frontal cortex and amygdala is really important for emotion and for planning actions
What you should know so far:

Instrumental conditioning
In contrast to Pavlovian conditioning, in instrumental conditioning the outcome is dependent on the animal’s behaviour
Needs to be a contingency between response and outcome (If I do this, that will happen)
Different types of contingencies and outcomes (we focus on positive reinforcement)
Johanson & Hall, 1979
Every time a device is turned on, the rat gets milk
They have to press a paddle
3 groups:
Rewarded group: Every time they press the paddle, they get milk, they pressed the paddle so much
Yoked control: They are in a cup right next to the rewarded pup → They get milk when the rewarded pup gets milk
It did not increase the rate of responding nearly as much
Deprived group: sat in a cup for 12 hours
They have the same responding rate as yoked control
They extend this: one paddle means milk, the other means no milk
Rewarded pup: lemon scented paddle leads to milk, clove means no milk
They learned the contingency! they learned the lemon paddle was the one to get milk
Yoke control did not get an increase
Why does animal perform the instrumental response? Ostlund & Balleine, 2009
10 days of training
Two very different types of food, when pressing bar on the left, one food, and on the right another kind
They press both bars equally whenever they want a certain food
Outcome devaluation → after stable rates of pressing, before testing them adding a whole bucket of a certain food, then put them back into the chamber to press bars
Number of lever presses per minute is lower for devalued outcome
Good evidence to assess that the animal is pressing the handle in order to get an outcome
Is it always the case?
10 vs 40 days of training
10 days of training respond more to the devaluation, but the overtrained group still want the same food after its devaluation
“habits” less sensitive to reward value of outcome
Once a behaviour becomes habitual, changing the outcome of the behaviour won’t change it
There are a range of factors that influence instrumental conditioning. One such factor is:
The delay between response and outcome (i.e., continguity)
Humans tolerate longer delay (paid fortnightly)
E.g. Smoking and sun-baking both have bad outcomes, the delay is too long to the bad outcome
Better late than never? not true, the sooner you give the outcome you are more likely to change the behaviour
Another factor that influences instrumental conditioning is the schedule of reinforcement
Continuous reinforcement schedule
Partial reinforcement schedule
Are two basic types of partial reinforcement schedules:
Interval and ratio
Either type can be fixed or variable

We can increase the ratio
Pigeon pecks, then gets food
Post reinforcement pause → you need a break
Lecture 5
Stress-enhanced Fear Learning
Nishimura
Two groups, both placed into a novel context
One group gets repeated shocks (stress)
Other group gets no shocks (no stress)
After some delay, animals places into completely different context, the animal can tell them apart
Animal are given a single, relatively mild shock
Animals should be afraid when they return to that environment if they have learned to associate it with that environment with the shocks
The more the animal has learned / are afraid, they will freeze
Both groups learned that the other environment is stressful
The group who had been stressed previously, have a much higher association of that environment with fear
Poulos et al
Gave some infant rats early life stress (repeated shocks in Context A), while control group did not get this stress
Then, in adulthood, tested for memory of that experience, then gave them a single shock in Context B
the animals shocked as infants, the higher the freezing
Babies forget that they have been shocked (infantile amnesia)
So NOT afraid of context A
That early stressful experience still led to increase association when they were shocked again as adults
This early stress is sensitisation.
“Natural” individual differences in fear expression?
Graham & Richardson (2016)
Trained adult rats to fear a context (Single shock US)
Some are very terrified after that one single shock
Some “learn” nothing → no reactions
Resilient animals
These two groups differed in levels of fibroblast growth factor-2 (FGF2) in the hippocampus
FGF2 is very important for neural development and neuroplasticity
More neuroplasticity, better at learning
Animals with low levels of freezing, show higher levels of FGF2
Illustrates individual differences despite having extremely similar life experiences
Graham, Zagic & Richardson (2019) - follow up
Looked at cue CS rather than context, and looked at rats and humans
Animals with low FGF2, freeze a lot more, they are much more afraid
In humans you see exactly the same thing
Relationship between naturally occurring neurotrophin and fear expression with FGF2 (and potentially resilience)
Extinction
Pairing of a CS+ with a shock US leads to the CS+ eliciting learned fear responses
Learning applies not only to the acquisition of new associations, but also to the rearrangement of existing ones
One samples of a situation where things have changed is extinction.
It is the decrease in the amplitude/frequency of a CR as a function on on-reinforced presentations of the CS
At a practical level, seems the same as habituation, but they are very very different. They have the same definition.
Acquisition
Extinction
CS-US
CS only
CS-US
CS only
CS-US
CS only
CS-US
CS only

Simplest explanation for extinction is that it is due to the “unlearning” of the CS-US association

During extinction, animal learns a new association (the first one is still there) but you learn something else.
E.g. Animal learns CS is no longer associated with US

Substantial evidence supports the competing memory hypothesis - especially the “3 Rs” of extinction
Spontaneous Recovery:
Recovery of responding that occurs when the CS is tested some time after the conclusion of extinction
US-CS Pairings, learn the association
Extinction training
Test some animals immediately after, a day later, week, etc
The longer you wait, the more likely the original CR response is to come back
Renewal
Recovery of responding when subject is tested in a context different from that where extinction occurred
Learning of CR is contextually modulated
Reinstatement
Recovery of responding when subject is tested after a non-signalled US presentation
All provide explanation that the original association does not “go away”, it can be retrieved.
Extinction is the basis for exposure based therapies
Goosens et al (2007)
Examined the effects of exposure therapy on spider phobia
Participants rated their fear of spiders, and also had measures of amygdala response to pictures of spiders taken
Then given single session of exposure therapy
Big difference in neuro activity of the brain
Amygdala is super fired up before therapy when seeing spiders
Amygdala basically the same level of fired up as the amygdala of the control group (those not scared of spiders)
Enhancement of extinction/exposure therapy?
One approach would be to reduce anxiety (ie., give an axiolytic)
That was the standard approach
However, if we accept that extinction (ie., exposure) is new learning, then maybe we need to give something that enhances learning
We want to enhance that new learning
A variety of neurotransmitters are involved in learning - but one key NT involved in learning is glutamate. Its primary receptor is the NMDA receptor.
Simplified molecular cascade of memory:

As noted by by Benito et al. the most important factor to consider when thinking about whether treatment is likely to be successful is the “degree to which exposures provide a relevant and/or potent learning experience”
Are lots of drugs that block the NMDA receptor (antagonists)
But we want something that increases activity at the NMDA receptor (agonist)
Are several NMDA receptor agonist, but work “too good”
However, recently a lab found that a partial NMDA receptor (D-cycloserine ; DCS) agonist might be effective without the negative side effects - enhances extinction of learned fear
Key point here is that this was an entirely near approach to pharmacologically enhancing extinction/exposure (ie., instead of reducing anxiety, the idea was to enhance the learning)
Walker et al. (2002) found that a single injection of DCS enhanced the long-term extinction of fear memory in rats
Final study
Trained adult male monkey to bar press to self administer alcohol (FR5 schedule - every 5th response = 30 seconds access to alcohol)
Extinction training - bar presses no longer gave access to alcohol; continued until animal responding less than 10% as much as the end of training (1 session per week)
Animal injected with DCS or vehicle prior to each extinction session
Those given DCS extinct twice as fast
Enhances the extinction of drug taking
They retrain them again until responding at same level as before
Takes a lot longer for the animals given DCS to reacquire the alcohol response
What you should know after this lecture
What are some factors that affect learned fear expression.
What is extinction?
How might we enhance extinction?