Lecture 1 What is learning? Learning is the acquisition of knowledge or skills through experience Typically, learning is revealed by a change in behaviour and the brain; these changes allow an animal to adapt to their environment Animals that are better learners might gain advantages Obtaining food Reproduction Avoiding predation Learning is important for dealing with these challenges Sometimes, however, an animal fails to learn or they might even learn the “wrong” thing (both of which can lead to problems) Mental health is the number ONE health problem in developed countries, in terms of years lost to disability (costs more than all cancers combined) US National Institute of Mental Health committed US $1.54 billion in 2017 in funding to reduce “the burden of mental illness and behavioural disorders through research on the mind, brain and behaviour” Learning is involved in some of these disorders Anxiety, and related, disorders (e.g., phobias, PTSD) Also, individuals with mental health problems may learn about the world in a different way Could affect how they interact with others, and how they respond to treatments Why do we study “animal” learning in psychology? We are animals. All animals have particular, similar challenges Maybe other animals solve challenges the same way we do Continuity of species - Darwin Focused on anatomical similarity (e.g. wings) Both species have to solve certain problems (e.g. bat vs pigeon), very different species but both have wings to solve the same problem Very very high degrees of similarity at some stage of development (e.g. embryos look so similar yet develop extremely differently) Similarities that we see across species, e.g. how we respond to predators Also in the brain, in neural processes that are very similar There is a molecular cascade of memories that happens in snails, monkeys, rats, to our knowledge even in humans All animals when they form memories experience this memory cascade The amygdala is important for emotional learning across several species Might the same principle (continuity of species) apply to learning? Helps animal adapt Instantiated in the brain Lots of potential applications Can help us understand: Our likes and dislikes (maybe many of those are learned early in development?) Anxiety disorders, and their treatment Rodents especially help us learn about how anxiety disorders are eased (CBT) Neural bases of learning and memory (drugs to improve memory?) Non-associative learning A type of learning where an animal’s behaviour/physiology/brain changes following repeated exposures to a stimulus Two types Habituation - a decrease in response amplitude or frequency as a consequence of repeated experience with a stimulus Is perhaps the most common type of learning Is stimulus specific Critically important for functioning in the world Allows the animal to avoid the familiar, focus on the new. We have limited attentional capacity, we are bombarded by sensory information. We need to ignore the familiar Heart rate orienting response Adult rats walking around, they hear a beep beep sound, first time they hear it, their heart rate goes down, but the more they hear it they respond less. Even if there is a time break, they will remember the sound and know not to respond Infants are not the same, they forget much more rapidly (infantile amnesia) Mutschler et al (2010) Expose pps to a classical piano piece for 40 mins, they ask them how arousing do you find this, how much do you like it, while pp is in an fMRI. Their initial liking response and arousal gets lower and lower as time goes on - they get bored. Amygdala response gets lower too Gandhi et al (2021), measured skin conductance responses to a repeatedly presented sound (a metronome) in a group of young adults that had been diagnosed with autism spectrum disorder (ASD) or not (labelled as neurotypical) ASD: Same magnitude initially as NT pps, but as time goes on, they don’t habituate, they keep responding to it They are learning about the world in a different way NT: They are responding 20% less than they were at the start Also applies to other types of stimuli e.g. in our species faces convey a lot of information, and we respond to a new face The neural response (in hippocampus) to a fearful faces habituates in non-diagnosed adults (Breiter et al., 1996) Big difference between NT and those with Schizophrenia, more activity in the latter’s brain. Non diagnosed people habituate. Hippocampus continues to pay attention to this person. Very simple type of learning can tell us about how all animals react to their ever-changing world. Further, it may give us insights into how individuals with a particular psychopathology react differently to the world Sensitisation - an increase in response amplitude or frequency as a consequence of repeated experience with a stimulus Lecture 2 Habituation (continued) Benito et al (2018) Benito et al (2018) examined habituation and treatment outcomes in a group of individuals diagnosed with pediatric OCD Looked at 100+ of patients who had received CBT for OCD Found single best predictor of treatment success in all aspects of treatment success (global improvement, reduced symptoms etc), was habituation of fear response Habituation increases, and OCD symptoms change Did not mattered how much you were exposed to it, but only that they were habituated to that fear Benito et al. : the most important factor to consider when thinking about whether treatment is likely to be successful is “the degree to which exposures provide a relevant and/or potent learning experience” Holz et al (2021) Measured the neural response to emotional faces (angry/fear) in the amygdala Participants were on average 25 years old, but had been apart of this study for entire life Parents signed up at birth (longitudinal study) At 3 months old, came into lab with mother, video-taped during a lab visit, and the amount of maternal interaction (i.e., “maternal stimulation”) was determined Classified maternal interactions as high (playing with kid, lots of love) or low maternal simulation (not as much attention to the child e.g. being on their phone) Results: Lifetime diagnosis of ADHD was weakly associated with habituation of the amygdala response (i.e., less habituation in those diagnosed with ADHD) The amount of maternal stimulation affected habituation but ONLY in high risk families Only did so in those from “high risk families” means that one or both of their parents had been diagnosed with a psychiatric condition, then high risk Level of maternal engagement can protect one from coming from a high risk family Remember, this measure of maternal stimulation was obtained when the 25 year old participants were only 3 months of age This indicates we can get a really early measure of who is high risk and even HIGHER risk Sensitisation An increase in response amplitude or frequency as a consequence of experience with a stimulus Sensitisation is more likely through strong stimuli, like electric shock Is not stimulus specific Can memory be “transferred” from one animal to another (even though the “other” animal didn’t have the relevant experience)? E.g. you get angry hearing a song you don’t like, and then get angrier and angrier every time they hear it Bedecarrats et al, 2018 A group of adult Alysia → 5 mild electric shocks to their tail 24 hours later, they get 5 more They are tested 24 hours, test is a water squirt to their siphon, how long does the siphon withdrawal for Other group is tested the exact same but without the shocks Result: Animals shocked stayed withdrawal 10s longer They took RNA from control and trained group Gave the RNA via injection to untrained animals Animals given trained RNA, had the near same reaction to those who experienced it, despite never having experienced the shocks

Boileau et al. (2006) Measured brain response to amphetamine (dopamine release in striatum) 3 doses of amphetamines via injections in healthy adults 2 weeks later, second injection Then 1 year later Brain is way more fired up after each dose Way more dopamine release after each time Summary In habituation/sensitisation, animals learn to recognise that event is familiar. But, they do NOT learn anything about the relation between that event and any other. Learned relationships between events linked in time and/or space are called associative relationships Pavlovian conditioning (associative learning) Animals learns that one stimulus predicts another stimulus (i.e., that goes with that) Originally studied digestion Unconditioned stimulus Food in example, usually a biologically relevant stimulus Unconditioned response Salivation, unlearned response in US Conditioned stimulus Bell in example, a neural stimulus that does not initially elicit the UR Conditioned response Salivation in example, response to the CS after learning has occurred

Important for our emotional reactions, as well as many other surprising things Lecture 3 Associative Learning Experimental “neuroses” Pavlov’s dog study Phase 1: the dog is brought down to lab and put into harness, periodically a circle is presented for 10 seconds, and at the end of the 10 seconds, it gets food, this is repeated. It should end up salivating when the circle is presented. Phase 2 (discrimination): dog is in a harness sees the circle and gets food, but it is also going to see an oval shape, when presented oval he does not get food. Dog still salivates at the oval. Overtime, the dog will learn to discriminate. Final phase: They see the circle and get food, but when they see another shape that looks exactly like the circle, they don’t get the food. This totally changed their emotional state, dog originally is placid and calm. It gets angry and neurotic. Pavlovian conditioning also very important for emotional reactions in humans Simple illustration of this is a study by Olsson & Phelps (2007) One stimulus (e.g. blue square on computer screen) predicted shock while a different stimulus (e.g., red square) predicted no shock. These two stimuli are referred to as the CS+ and the CS- respectively Measured Electrodermal Activity (EDA AKA skin conductance) in skin to the new stimuli CS+ starts to elicit a state of fear (skin conductance response increases), pps showed more reaction to CS+, they also measured neural activity in the amygdala (MRI) Amygdala had much more activity when presented with the CS+ We can learn about the associations between two stimuli even if we don’t experience them The highly social word that humans live in provides many opportunities to observe others’ emotional reactions to various stimuli Observational (vicarious) learning - learning that occurs from simply watching another (i.e., no direct experiences of US) Olsen & Phelps (2007) → extended experiment, participants watched other participants seeing the blue and red square Measured EDA, pps in the two conditions exhibit comparable learning! Also measured neural responses to the two stimuli (focused on the amygdala), showed exact same response despite never having experienced it Pavlovian conditioning thought to be especially important for the learning of emotional reactions (e.g., what we like) Long-standing view in psychology is that early experience have an especially pronounced impact Brunjes & Alberts’ study of huddling behaviour circular chamber: one side has an anesthetised adult gerbil and on the other side is an anesthetised adult rat (they are laying down not moving because anesthetised), they dropped 10 day old rats and 15 day old rats into the circular chamber to observe 10 day old rats 50% with rat and the gerbil 15 days old, 80% with rat, 20% gerbil This is referred to as a transition from physiological to filial huddling At first, they just want a warm body so they want to stay warm with whoever, but then 5 days later, they want to stay warm but also want to do it with rats Alberts and May asked whether this transition was due to “nature” or was it due to “experience” Rats suck their mother’s nipple and sometimes milk comes out US = food, CS = smell Nursing mothers were sprayed with either “wild musk” or saline (odourless/ normal smelling) Rats tested from 5, 10, 15 and 20 days old 5-10 days don’t care, just want warm body (50-50) 15-20 days the change occurs, if rats were raised by a rat that smelled like wild musk, then they spend 80% of their time with a rat that smells like wild musk. If reared by normal smelling rat, they spend 80-90% with normal smelling rat Suggests the transition from physiological to filial huddling is because of Pavlovian conditioning, they are conditioned by smell Fillion & Blass (1986), rats were reared by a mum that smelled “normal” or of “citral” (odour sprayed on her ventrum each day) Pups were separated from mum at 21 days of age, and male offspring tested when they were 100 days off age Test consisted of being placed in a chamber with a female that was in heat Prior to test, some females sprayed with citral, others with saline They tested the time it took for the rats to ejaculate, depending on the smell of the female When rats ejaculate quicker when the rat smells like their mother rat Olfactory learning reported by Sullivan In a typical experiment, she pairs an odour CS with mild electric shock US in rats of various ages (has two control groups at each age). She then measures odour preference (5 test trials) Ages 6, 12 and 20 There are typically two control groups 1: Odour, no shock 2: Odour 5 times and shock 5 times but it is random, not predictable At older ages, they avoid the odour when paired with shock 6 day old, they love the electric shock? They keep going for the CS odour Explains child abuse, they are still attached to the abuser and love them, potentially this learning at a young age about a primary caregiver - even if there are bad things associated with them Tiffany Field: Massage therapy in premature human infants She did an experiment to massage the baby 15 mins a day First experiment, just nurses Grew fasters, discharged from hospital 6 days sooner; average hospital savings of $10k per infant = 4.7 billion dollars per year in US alone (Field, 2004) Second, mothers Third, depressed mothers Allowing the depressed mother to rub the baby helps the post-partum depression symptoms Over 30% higher survival rate for babies rubbed than non-rubbed Lecture 4 Olfactory Learning - Sullivan (continued) Learn that odour predicts shock The youngest age group learn a preference for that odour, which is very puzzling The findings change if you train the rat in the presence of their mother Mother is not getting shocked, but just present 12 day old will now prefer the odour that predicts a shock Tottenham et al (2019) Tested 3-5 year old children They were first trained that one shape (blue square) sometimes predicted an aversive, loud scream; this stimulus is referred to as the CS+ A different shape (purple triangle) never predicted the aversive US; this stimulus referred to as CS- Some trained alone and others in presence of parent After training they are tested In a room behind a door there is a playroom, child is told that there will be a playhouse inside There are two arms in the room

They are told they can go into either arm of the room, they are told both has a prize 3-5 year old child sees a certain geometric shape, and hear a scream Parents have left at this point When kids were not trained by their parents, they preferred the CS- 55% of time, kids trained with mother prefer the CS+ Pavlovian conditioning and Criminality Gao et al., 2010 Trained ~900 kids (3 yrs old) across 2 years on a fear conditioning task (i.e., CS+ = shock, CS- = no shock) Was done in Mauritius Then waited for 20 years for the data of interest! 137 of those kids were convicted of serious crimes (rape, assault, murder etc) 750 were not convicted They looked at the demographics of all the kids Tried to match non convicted and convicted kids, this was control group 200 kids that were not convicted but had similar demographic to convicted kids The children who grew up to be convicted of serious crimes, they did not exhibit the learning, the fear response to the CS+ - no increase in skin conductance Birbaumer et al (2005) Fear conditioning in psychopaths Devoid of emotional reactions All male pps, all incarcerated individuals, all found guilty of serious crimes (murder, repeated rape etc) Psychiatrists had diagnosed some as psychopaths Some were not psychopaths (control) CS+ and CS- = neutral faces US = painful pressure Measured subjects’ ratings of CS-US contingencies, SCR to CS+ and CS-, and neural activity to each CS Contingency ratings → how likely it is that I will experience pain when certain face shown, no difference between both groups, they both learned Skin conductance responses extremely different, increase in SCR for control group for the CS+ The psychopaths do not respond to it at all, they do not experience the emotional reaction They also measured neural activities, in healthy controls, there are parts of the brain that are much more active when seeing the CS+, but the activity is not there for psychopaths (less activation in the amygdala and the orbitofrontal cortex) The psychopaths still experience the same pain, but they don’t learn! Orbital frontal cortex and amygdala is really important for emotion and for planning actions What you should know so far:

Instrumental conditioning In contrast to Pavlovian conditioning, in instrumental conditioning the outcome is dependent on the animal’s behaviour Needs to be a contingency between response and outcome (If I do this, that will happen) Different types of contingencies and outcomes (we focus on positive reinforcement) Johanson & Hall, 1979 Every time a device is turned on, the rat gets milk They have to press a paddle 3 groups: Rewarded group: Every time they press the paddle, they get milk, they pressed the paddle so much Yoked control: They are in a cup right next to the rewarded pup → They get milk when the rewarded pup gets milk It did not increase the rate of responding nearly as much Deprived group: sat in a cup for 12 hours They have the same responding rate as yoked control They extend this: one paddle means milk, the other means no milk Rewarded pup: lemon scented paddle leads to milk, clove means no milk They learned the contingency! they learned the lemon paddle was the one to get milk Yoke control did not get an increase Why does animal perform the instrumental response? Ostlund & Balleine, 2009 10 days of training Two very different types of food, when pressing bar on the left, one food, and on the right another kind They press both bars equally whenever they want a certain food Outcome devaluation → after stable rates of pressing, before testing them adding a whole bucket of a certain food, then put them back into the chamber to press bars Number of lever presses per minute is lower for devalued outcome Good evidence to assess that the animal is pressing the handle in order to get an outcome Is it always the case? 10 vs 40 days of training 10 days of training respond more to the devaluation, but the overtrained group still want the same food after its devaluation “habits” less sensitive to reward value of outcome Once a behaviour becomes habitual, changing the outcome of the behaviour won’t change it There are a range of factors that influence instrumental conditioning. One such factor is: The delay between response and outcome (i.e., continguity) Humans tolerate longer delay (paid fortnightly) E.g. Smoking and sun-baking both have bad outcomes, the delay is too long to the bad outcome Better late than never? not true, the sooner you give the outcome you are more likely to change the behaviour Another factor that influences instrumental conditioning is the schedule of reinforcement Continuous reinforcement schedule Partial reinforcement schedule Are two basic types of partial reinforcement schedules: Interval and ratio Either type can be fixed or variable

We can increase the ratio Pigeon pecks, then gets food Post reinforcement pause → you need a break Lecture 5 Stress-enhanced Fear Learning Nishimura Two groups, both placed into a novel context One group gets repeated shocks (stress) Other group gets no shocks (no stress) After some delay, animals places into completely different context, the animal can tell them apart Animal are given a single, relatively mild shock Animals should be afraid when they return to that environment if they have learned to associate it with that environment with the shocks The more the animal has learned / are afraid, they will freeze Both groups learned that the other environment is stressful The group who had been stressed previously, have a much higher association of that environment with fear Poulos et al Gave some infant rats early life stress (repeated shocks in Context A), while control group did not get this stress Then, in adulthood, tested for memory of that experience, then gave them a single shock in Context B the animals shocked as infants, the higher the freezing Babies forget that they have been shocked (infantile amnesia) So NOT afraid of context A That early stressful experience still led to increase association when they were shocked again as adults This early stress is sensitisation. “Natural” individual differences in fear expression? Graham & Richardson (2016) Trained adult rats to fear a context (Single shock US) Some are very terrified after that one single shock Some “learn” nothing → no reactions Resilient animals These two groups differed in levels of fibroblast growth factor-2 (FGF2) in the hippocampus FGF2 is very important for neural development and neuroplasticity More neuroplasticity, better at learning Animals with low levels of freezing, show higher levels of FGF2 Illustrates individual differences despite having extremely similar life experiences Graham, Zagic & Richardson (2019) - follow up Looked at cue CS rather than context, and looked at rats and humans Animals with low FGF2, freeze a lot more, they are much more afraid In humans you see exactly the same thing Relationship between naturally occurring neurotrophin and fear expression with FGF2 (and potentially resilience) Extinction Pairing of a CS+ with a shock US leads to the CS+ eliciting learned fear responses Learning applies not only to the acquisition of new associations, but also to the rearrangement of existing ones One samples of a situation where things have changed is extinction. It is the decrease in the amplitude/frequency of a CR as a function on on-reinforced presentations of the CS At a practical level, seems the same as habituation, but they are very very different. They have the same definition. Acquisition Extinction CS-US CS only CS-US CS only CS-US CS only CS-US CS only

Simplest explanation for extinction is that it is due to the “unlearning” of the CS-US association

During extinction, animal learns a new association (the first one is still there) but you learn something else. E.g. Animal learns CS is no longer associated with US

Substantial evidence supports the competing memory hypothesis - especially the “3 Rs” of extinction Spontaneous Recovery: Recovery of responding that occurs when the CS is tested some time after the conclusion of extinction US-CS Pairings, learn the association Extinction training Test some animals immediately after, a day later, week, etc The longer you wait, the more likely the original CR response is to come back Renewal Recovery of responding when subject is tested in a context different from that where extinction occurred Learning of CR is contextually modulated Reinstatement Recovery of responding when subject is tested after a non-signalled US presentation All provide explanation that the original association does not “go away”, it can be retrieved. Extinction is the basis for exposure based therapies Goosens et al (2007) Examined the effects of exposure therapy on spider phobia Participants rated their fear of spiders, and also had measures of amygdala response to pictures of spiders taken Then given single session of exposure therapy Big difference in neuro activity of the brain Amygdala is super fired up before therapy when seeing spiders Amygdala basically the same level of fired up as the amygdala of the control group (those not scared of spiders) Enhancement of extinction/exposure therapy? One approach would be to reduce anxiety (ie., give an axiolytic) That was the standard approach However, if we accept that extinction (ie., exposure) is new learning, then maybe we need to give something that enhances learning We want to enhance that new learning A variety of neurotransmitters are involved in learning - but one key NT involved in learning is glutamate. Its primary receptor is the NMDA receptor. Simplified molecular cascade of memory:

As noted by by Benito et al. the most important factor to consider when thinking about whether treatment is likely to be successful is the “degree to which exposures provide a relevant and/or potent learning experience” Are lots of drugs that block the NMDA receptor (antagonists) But we want something that increases activity at the NMDA receptor (agonist) Are several NMDA receptor agonist, but work “too good” However, recently a lab found that a partial NMDA receptor (D-cycloserine ; DCS) agonist might be effective without the negative side effects - enhances extinction of learned fear Key point here is that this was an entirely near approach to pharmacologically enhancing extinction/exposure (ie., instead of reducing anxiety, the idea was to enhance the learning) Walker et al. (2002) found that a single injection of DCS enhanced the long-term extinction of fear memory in rats Final study Trained adult male monkey to bar press to self administer alcohol (FR5 schedule - every 5th response = 30 seconds access to alcohol) Extinction training - bar presses no longer gave access to alcohol; continued until animal responding less than 10% as much as the end of training (1 session per week) Animal injected with DCS or vehicle prior to each extinction session Those given DCS extinct twice as fast Enhances the extinction of drug taking They retrain them again until responding at same level as before Takes a lot longer for the animals given DCS to reacquire the alcohol response What you should know after this lecture What are some factors that affect learned fear expression. What is extinction? How might we enhance extinction?