1/65
Looks like no tags are added yet.
What is learning?
Long-term change in behaviour/state brought about by experience (involves changing synaptic strengths); a collection of different techniques, procedures + outcomes that produce changes in an organism’s behaviour
Can be conscious or unconscious
Can start out explicit + become implicit over time (e.g. driving a car)
Way behaviour comes about (physiologically) is different
Same action is performed more quickly
Different action is performed
How are memory and learning linked?
Learning produces memories
The existence of memories implies that knowledge was acquired, that experience was registered + recorded in the brain, or that learning has taken place
What is habituation?
General process in which repeated or prolonged exposure to a stimulus results in a gradual reduction in responding
Simple form of learning → an experience results in a change in the state of the learner BUT this is usually impermanent (i.e. a person will exhibit the og reaction if enough time has gone by
Leaton (1976) → used rats and tones to question how robust habituation to a loud sound is
Sensitisation = presentation of a stimulus leads to an increased response to a later stimulus
What is Hebbian learning?
“When an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.” (Hebb, 1949)
AKA neurons that fire together (at the same time), wire together (firing is more likely to happen)
What does long-term potentiation involve?
Increased release of neurotransmitters + changes in receptor density
Not all synaptic connections are strengthened (may fire at different circumstances)
In hippocampus → induced more rapidly + lasts for a long time
After LTP → more depolarisation effect → bigger ESPS
Action potential isn’t changed by the learning process only the efficiency/likelihood that it'll be elicited → more action potentials produced but same magnitude

Pre-synaptically, what happens after long-term potentiation?
Releases more neurotransmitters → more to bind with on post-synaptic cell

Post-synaptically, what happens after long-term potentiation?
Produces more receptors/channels for the neurotransmitters to bind to

What are long-term potentiation effects dependent on? What is this knowna as?
Timing → spike timing dependent plasticity
How is long-term potentiation tested?
Hippocampal slice → patch clamp to pre and post synaptic neurons → deliver a controlled stimulus to the pre-synaptic cell → axon fires → release neurotransmitters → measure what happens at post synaptic cell (baseline) → deliver stimuli to both cells at the same time → fire at the same time (shows stronger EPSP) → post-test (deliver stimuli to pre-cell + measure at post-cell)
What is the simplest form of learning (which is produced by long-term potentiation)?
Classical conditioning (stimulus evokes a response because of being paired with a stimulus that naturally evokes a response) → CS-US timing, STDP
Concern involuntary, reactive behaviours
US → stimulus that elicits a reflex or innate response (e.g., food)
UR → reflex or innate response elicited by the US (e.g., salivation)
CS → neutral stimulus that does not initially elicit a reflex or innate response (e.g., bell)
CR→ reflex or innate response elicited by CS after CS is paired with US (e.g., bell-salivation)

What are the 5 phases/basic principles of classical conditioning?
Acquisition
Second-order conditioning
Extinction
Spontaneous recovery
Generalisation
What is acquisition in classical conditioning?
Phase of classical conditioning when the CS and the US are presented together
Gradual increase in learning → starts low, rises rapidly and then slowly tapers off

What is second-order conditioning?
Conditioning where the US is a stimulus that acquired its ability to produce learning from an earlier procedure in which it was used as a CS
E.g. Pavlov repeatedly paired a new CS, a black square, with the now-reliable tone → dogs produced a salivary response to the black square even though it had never been directly associated with the food
RWA
Explain why some people desire money to the point of hoarding (initially used to purchase objects producing gratifying outcomes → becomes linked with this)
What is extinction in classical conditioning?
Gradual elimination of a learned response that occurs when the CS is no longer followed by the US
Behaviour declines abruptly + continues to drop until the learned response is no longer exhibited

What is spontaneous recovery?
Tendency of a learned behaviour to recover from extinction after a rest period
Takes place even though there have not been any additional associations between the CS + US
Ability of the CS to elicit the CR was weakened BUT it wasn’t eliminated → shows that the connection still exists
How can spontaneous recovery be explained in terms of neural activity?
Excitatory association = increases likelihood of a response
Inhibitory association = decreases the likelihood of a response
Pavolv's analysis started with the assumption that every stimulus activates its own unique area/set of neurons within the brain (a centre → cortical reps of the stimulus)
Whenever activity in one centre of the brain is closely followed by activity in another, the neural connection between the two centres will be strengthened
During extinction a new (inhibatory) connection is formed (becomes progressively stronger throughout extinction process) → activity in the 1st centre led to a reduction in activity in the 2nd centre
Excitatory + inhibatory connections are equally strong at the completion of extinction BUT inhibatory connection is more fragile → more easily disrupted by the passage of time (BUT contested e.g. Bouton, 2002)
What is generalisation? How does it relate to discrimination?
Increase in responding to a stimulus because of its similarity to a CS that was paired with a US
The less the new stimulus resembles the CS, the less conditioned responding is observed
Discrimination = the capacity to distinguish between similar but distinct stimuli
By responding to the new stimulus used during generalisation testing, the organism demonstrates that it recognises the similarity between the original CS + the new stimulus
By displaying diminished responding to that new stimulus, it also tells us that it notices a difference between the 2 stimuli
The more discrimination/generalisation is shown, the less the other is shown → training can modify the balance between the 2
What is STDP?
Pre-synaptic cell fires just before the post-synaptic cell → LTP (positive increase in EPSP → strength of connection/synapse got stronger)
Pre-synaptic cell fires after the post-synaptic cell (e.g. food first, bell second) → strength of connection/synapse gets weaker (long-term depression)

What is the Hebbian explanation of classical conditioning?
Bell isn't initially connected to saliva → artificially causes CS + CR to fire at the same time → wire together + connection becomes stronger

What can Pavlovian and Habbian learning be used to explain?
Discrete learning trials (learn associations between stimuli + rewards)
What are the cognitive elements of classical conditioning?
Zener (1937) → recreated Pavlov's experiment → saw increase in salivation + dogs turned towards the tray where the food was delivered in expectation (realise that CS signals food)
Rescorla + Wagner (1972) → CC is a cognitive process
Pavolv's dogs formed an expectation that the buzzer would be followed by good → salivation (contingency = expectation about how well the CS signals the appearance of the US)
CS needs to reliably predict the UCS → blocking (when a stimulus has been prepared with a natural stimulus to produce a CR, this experience blocks the capacity to learn other associations when the stimulus is present)
Shows we take into consideration expectations when learning new associations
Öhman + Soares (1998) → discrimination training (people learn to respond only to the stimulus that signals the US)
Masking stimulus effectively removes the opportunity for people to have time to become consciously aware of them
Conditioning can occur at a conscious or unconscious level
What is some evidence of the evolutionary elements of classical conditioning?
Conditioned food aversions
Conditioned food preferences
Bio preparedness
What are conditioned food aversions?
Any species that forages or consumes a variety of foods needs to develop a mechanism by which it can learn to avoid any food that once made it ill
Requires:
Rapid learning that occurs in 1 or 2 trials → prevents death from eating a toxic substance repeatedly
Conditioning should take place over long intervals → toxic substances often don’t cause illness immediately, so the organism would need to form an association between food and the illness over a longer term
Develop the aversion to the smell or taste of the food rather than its ingestion → more adaptive to reject a potentially toxic substance based on smell alone than it is to ingest it
Learned aversions should occur more often with novel foods than familiar ones → not adaptive for an animal to develop an aversion to everything it has eaten on the particular day it got sick
Garcia + Koelling (1966)
RWA
Technique for dealing with an unanticipated side effect of radiation + chemotherapy (Broberg + Bernstein, 1987 → given unusual food at the end of the last meal before undergoing treatment)
What are conditioned food preferences?
Sclafani (1995) → preference for nutritive substance
Ackroff (2005) → conditioned food preferences can be produced by sources of fat, such as corn oil and safflower oil
What is biological preparedness?
Propensity for learning particular kinds of associations over others (some behaviours are relatively easy to condition in some species but not others)
E.g. phobias → humans have a biological preparedness to develop phobias of situations that, in our evolutionary past, were potentially dangerous to survival (Öhman + Mineka, 2001)
A species that is relatively physically vulnerable + has poor night vision needs to be wary of predators that lurk in the night
A species that spends a good bit of time in trees will live longer if it develops a healthy appreciation of the dangers of falling
Which area of the brain is critical for emotional conditioning?
Amygdala (central nucleus)
Fear conditioning → behavioural (e.g. freezing) + physiological (e.g. fight or flight) components occur but are elicited by CS
Produces these through 2 distinct connections with other parts of the brain → if connections linking the amygdala to the lateral part of the hypothalamus are disrupted/severed, neither components occur (LeDoux, 1988)
What was the Little Albert experiment?
The sight of a white rabbit, a seal fur coat + a Santa Claus mask produced the same kinds of fear reactions in the infant
Aims:
Show that relatively complex reactions can be conditioned using Pavlovian techniques
Emotional responses (e.g. fear + anxiety) could be produced by CC → don't need to be the product of deeper, unconscious processes/early life experiences
Conditioning can be applied to humans as well as animals
Watson argued that learning + the environment determine behaviour (more than genetics + personality)
What are the concepts of classical conditioning and extinction helpful for?
Anxiety disorders = marked by excessive fear, worry, anxiety, often about situations or things
Phobias
OCD
PTSD
Some effective treatment strategies use methods derived from classical conditioning → extinction is a key mechanism of action in exposure therapy
Exposed to a weak version of the stimulus such that it will elicit the response BUT not the full anxiety response
Do overdose cases have substantially higher concentration of drugs in their system (compared to non-overdose cases)?
No → “Blood morphine concentrations in fatal cases are frequently below those of intoxicated heroin users, or users who died due to causes other than drug toxicity” (Darke, 2014)
Victims are often experienced at taking drugs, the dose is not larger than what they usually take + the deaths occur in unusual settings
How can drug overdose be understood through classical conditioning?
Situational cues become associated with drug intake (setting = CS) → drug metabolism (tolerance) is enhanced
Brain reacts to the heroin by secreting neurotransmitters that counteract its effects (produces drug tolerance → form of physiological habituation) → protective physiological response becomes a part of the CR
If the drug is taken in a different situation (situational cues are absent) → lowers tolerance → users are more vulnerable to overdose (metabolic response is triggered)
New environment → CS is altered → physiological compensatory CR either doesn't occur or is substantially decreased → usual dose becomes an overdose
Converging evidence from interviews of addicts + non-human animal studies with precise control over context
Based on a classical conditioning, how can drug overdose be reduced?
Drug consumption rooms
Provide a reliable, controlled context to reduce overdose likelihood
Reduce harm caused by drug misuse
Clean needles
Access to emergency care in case of overdose
Access to addiction treatment
Can be useful in areas where drug misuse is common
Cue exposure therapy
Exposed to drug-related cues without being given the usual dose of the drug itself → extinction of association between the contextual cues and the effects of the drug
Encountering familiar drug-related cues will no longer result in the compensatory response linked to withdrawal symptoms → easier for someone recovering from substance use disorder to remain abstinent (Siegel, 2005)
Give an example of the use and efficacy of drug consumption rooms
Large-scale assessment of a medically-supervised injecting centre in Australia
Scale: 5 years (2007-2011), avg >5600 visits per month (70k visits/year)
Managed >3200 overdose events with no deaths onsite
40% of users had no previous interaction with any form of drug treatment
~400 referrals for drug treatment per year (plus another 400-500 referrals for other types of treatment per year)
~50% decrease in public injecting + discarded needles
No increase in local crime or drug use
Saved at least $658k/year over providing similar health outcomes through other means in the health system

What did EL Thorndike (1898) argue about the environment?
It can elicit voluntary behaviours → learning takes place in contexts, not in the free range of any plausible situation
Studied instrumental behaviours → require an organism to do something, solve a problem or manipulate elements of its environment
E.g. puzzle box → door would open when concealed lever is pushed in the right way
Over successive trials the cats learned to escape from the box more quickly → gradual + uneven progress
Suggests that the correct response was being automatically strengthened by the food that followed it NOT insight/understanding
Behaviour of the animal determined what happened next → unlike Pavlov (dogs were fed regardless)
Law of effect = behaviours that are followed by a satisfying state of affairs tend to be repeated + those that produce an unpleasant state of affairs are less likely to be repeated
What is operant (instrumental) conditioning?
Learning happens through naturally exhibited behaviours being reinforced → use the technique of successive approximations/shaping to obtain new behaviours
Active, voluntary behaviours
Need education of desired behaviour alongside reinforcement
Primary reinforcer = effective form birth for all members of a species, don't need to be learned as all members need them to survive + will behave accordingly to gain access to them
Often gain function from basic bio mechs
Secondary/conditioned reinforcer = not innately effective, acquire this through experience + learning by being paired with primary reinforcers (CC)
As long as the CS–US link is maintained, the secondary reinforcers + punishers can be used to modify and control behaviour
Broken links → extinction procedure → lose functions
Gradually increase how close animal has to get to the behaviour we want before the animal is reinforced
Dominant approach to modelling human decision making in a wide variety of contexts
Successes of AI/ML are trained using reinforcement learning
Games (TD-Gammon, DQN Atari, AlphaGo)
Real-time action control
Who coined operant conditioning?
All these emitted behaviours operated on the environment in some manner → produced a change in that environment → strengthened the behaviours that produced them
Operant conditioning chamber/Skinner's box
Rats → pigeons (more easily trained, persistent, little sleep, good vision to present a wide range of stimuli)
Presenting food or electric shock → reinforcement vs punishment
+/- punishment/reinforcement
Most behaviour is under stimulus control → develops when a particular response occurs only when the appropriate stimulus is present
Shows discrimination + generalisation
Watanabe (1995) → discriminated between painters + generalised across painters as long as they were from the same artistic tradition
Stimulus control + its ability to foster stimulus discrimination and stimulus generalisation, is effective even if the stimulus has no meaning to the respondent
What is the premack principle?
Discerning which of two activities someone would rather engage in means that the preferred activity can be used to reinforce a nonpreferred one
Important to establish a hierarchy of behaviours for an individual in order to determine which kinds of events might be maximally reinforcing
What is the neutrality of reinforcers?
A stimulus or event will be relatively reinforcing based on a host of factors, many of which are specific to the individual
The effectiveness of particular stimuli can be manipulated
What is extrinsic reinforcement?
Rewards that come from external sources
Rewards can sometime decrease performance of a behaviour because the extrinsic reinforcement undermines the reasons why people engage in behaviour in the first place
People often engage in activities for intrinsic rewards (e.g. pleasure)
How did Lepper and Greene (1978) contribute to operant conditioning?
Children were given coloured pens and paper and asked to draw whatever they wanted
Some received a good player award → spent more time at the task than others
When the experimenters stopped handing out the Good Player Awards to the first group, the amount of time the children spent drawing drop
Explanation
Change in motivation → reward undermined children's original intrinsic interest
Over justification effect = external rewards undermine the intrinsic satisfaction of performing a behaviour
Is a reward necessarily harmful when there is intrinsic interest?
No → deoends on what the reward is + how it’s used
Material rewards have a greater danger of undermining (Cameron and Pierce, 1996; Deci et al., 1999)
What is a discriminative stimulus?
Stimulus that’s associated with reinforcement only in a particular situation
What is acquisition in operant conditioning?
Steady rate of response until reinforced
Reinforcement is followed by a pause in responding
Response starts slow + speed up as the time that a reinforcer is delivered approaches → scalloping
What is extinction in operant conditioning?
Reinforcement not delivered → behaviour eventually stops
Fixed ratio shows most resistance to extinction
The effect of reinforcement depends on how often a behaviour is reinforced → extinction doesn't always occur but may lead to resilience (e.g. continuing to sell cars even when someone doesn't buy after doing a test drive)
What are the schedules of reinforcement?
Continuous = all responses are reinforced
Partial
Interval schedules = reinforcement is based on how much time has elapsed since the previous reinforcement
Ratio schedules = reinforcement is based on how many responses have been made (specified ratio of responses to reinforcers)
All ratio schedules encourage high + consistent rates of responding because the number of rewards received is directly related to the number of responses made
Both schedules provide intermittent reinforcement → only some of the responses made are followed by reinforcement
All produce behaviour that is much more resistant to extinction than a continuous reinforcement schedule does

What is a fixed interval schedule?
Reinforcement will become available when a fixed time period has elapsed following the previous reinforcement
The 1st response after this interval will produce the reinforcer
Reinforcement is not necessarily delivered when the fixed time has elapsed → reinforcement becomes available at that time, but it will be delivered only when a response is made
Skinner's box → show little responding right after the presentation of reinforcement, but as the next time interval draws to a close, they show a burst of responding
FI scallop
What is a variable interval schedule?
Reinforcement will become available when a time period has elapsed following the previous reinforcement
Length of the waiting period varies from one reinforcer to the next (on average)
Typically produce steady, consistent responding → the time until the next reinforcement is less predictable
What is a fixed ratio schedule?
Reinforcement will be delivered after a specific number of responses have been made
One schedule might present reinforcement after every fourth response, while a different schedule might present reinforcement after every 20 responses
E.g. free drink after a set number of purchases
Continuous reinforcement = presenting reinforcement after each response
Is possible, in principle, to know when the next reinforcer is due
What is a variable ratio schedule?
Reinforcement will be delivered after a specified average number of responses have been made
E.g. in casinos
What happens when a schedule is more irregular and intermittent?
More difficult for an organism to detect when it has actually been placed on extinction
Under conditions of intermittent reinforcement, all organisms will show considerable resistance to extinction and continue for many trials before they stop responding
Weird (2005) → observed this effect in children
Intermittent reinforcement effect = operant behaviours that are maintained under intermittent reinforcement schedules resist extinction better than those maintained under continuous reinforcement (relationship between intermittent reinforcement schedules and the robustness of the behaviour they produce)

Schedules of reinforcement (diagrams)

How does operant conditioning compare to Pavlovian and Hebbian learning?
OC occurs in continuous time → reflects expectations about future rewards
Learning predictions about rewards from different states
Value of different states → select actions that max value
CC → number of learning trials is important
OC → pattern with which reinforcements appear is crucial
What did Skinner argue about Pavlov and Thorndike’s experiments?
Trial-by-trial experiments of P + T were artificial → behaviour rarely occurs in fixed frameworks where a stimulus is presented + then PPTs engage in some activity
Continuously acting + behaving
The world around us reacts in response to our actions
Most of our behaviours are the result of shaping = learning that results from the reinforcement of successive approximations to a final desired behaviour
Behaviour that gets incrementally closer to the overall desired behaviour until the overall sequence of behaviour is produced reliably
How did Skinner explain superstitious behaviour?
Pepeating behaviours than had been accidentally reinforced (accidental correlation NOT causation)
Responsible for human superstitions
BUT bird experts argued that the ‘superstitious’ behaviours were all e.gs of food-searching behaviour in pigeons → non-accidental (Timberlake + Lucas, 1985)
What sort of distribution does operant conditioning have?
Can influence neurons all over the brain → neuromodulation (vs info processing)
How can operant conditioning be applied to motor impairments?
Taub (2002) → learned non-use (reinforced behaviour or not using the limb → further impairment)
Constraint-induced movement therapy
Constrain unaffected limb to force use of the affected limb → e.g. put in a sling
Overcomes non-use → process of shaping to gradually improve performance of affected limb

What are the neural elements of operant conditioning?
Pleasure centres
Neurons in the medial forebrain bundle
How are pleasure centres involved in operant conditioning?
First hint of how specific brain structures might contribute to reinforcement
James Olds (1956) → inserted electrodes into parts of a rat's brain + allowed control of the electric stimulation of its own brain
Some brain areas, particularly in the limbic system, produced intensely positive experiences
If drugs that block the action of dopamine are administered to the rats, they cease stimulating the pleasure centres (Stellar et al., 1983)
How are the neurons in the medial forebrain bundle involved in operant conditioning?
MFB = pathway that meanders its way from the midbrain through the hypothalamus into the nucleus accumbens
Most susceptible to stimulation that produces pleasure
All dopaminergic
BUT not the basis for liking → dopamine has more to do with pursuit (incentive for potential rewards)
If the action of dopamine-producing cells is knocked out in genetically modified mice, it does not change the pleasure that they derive from food (Cannon and Palmiter, 2003)
What is the evidence for the neural elements of operant conditioning?
Drugs (e.g. cocaine, amphetamine + opiates) activate these pathways + centres (Moghaddam + Bunney, 1989)
BUT dopamine-blocking drugs dramatically diminish their reinforcing effects (White + Milner, 1992)
FMRI studies → increased activity in the nucleus accumbens in straight men looking at pictures of attractive women (Aharon, 2001) + in individuals who believe they are about to receive money (Knutson, 2001)
Human neuroimaging → similar circuitry is activated by diverse pleasures, indicating a common neural currency shared by all (Berridge and Kringelbach, 2015)
Rats that are given primary reinforcers (e.g. food, water, sex) show increased dopamine secretion in the nucleus accumbens, BUT only if the rats are hungry, thirsty or sexually aroused (Damsma, 1992)
According to Tolman, what are the cognitive elements of operant conditioning?
Animal establishes a means-end relationship → the conditioning experience produced knowledge or a belief that, in this particular situation, a specific reward (the end state) will appear if a specific response (the means to that end) is made
Stimulus doesn't directly evoke a response but establishes an internal cognitive state → produces a behaviour
Focuses on part between S-R
1930s + 40s → conducted studies focusing on latent learning (something is learned but it is not manifested as a behavioural change until sometime in the future) + cognitive maps
Rats completing a complex maze: control (no reinforcement), regular reinforcement, control group for 1st 10 days, then reinforced for 7 (showed evidence of latent learning)
Formed a cognitive map = mental rep of the physical features of the environment
According to Vander Wall (1982), what are the cognitive elements of operant conditioning?
2 birds hide seeds in a fenced-in enclosure with landmarks + then released these birds and 2 others (who hadn't hidden the seeds)
Birds were not just searching at random, they were using landmarks to remember where they had hidden the seeds + then went there with almost unerring accuracy
Suggests they form extraordinarily detailed cognitive maps of their environments, containing the locations of thousands of caches in relation to the landmarks that define them
What are the evolutionary elements of operant conditioning?
Foraging animals have well-developed spatial reps = the capacity to encode, process + store info about the shape + layout of the physical environment → allow them to construct cognitive maps + search environment efficiently
Brelands
Each species is bio predisposed to learn some things more readily than others + to respond to stimuli in ways that are consistent with its evolutionary history (Gallistel, 2000)
Thomas (1981)
A rat that pressed the bar in Skinner's box regularly would lose out on every other reinforcement
The rats more often opted for immediate rewards → consequences of evolution (acting too readily on the association between bar pressing + food delivery backfired as a LT adaptive strategy)
What is reinforcement learning?
The value of a particular state (Vs) is a function of expected future rewards (Rt) that stem from being in that state


What is task state representation?
Mice can learn abstract rule: “rewards appear in A→B→C→D order”
Then rapidly apply that rule to new A/B/C/D locations
Neurons in medial frontal cortex tend to keep their task state representation across tasks
Medial frontal cortex reps task state as abstract location

How is the OFC involved in value-based decision-making?
Decision subspace = decisions/actions are complex + context-specific
OFC provides a simpler, more general “value” (aka utility, reward) rep for making decisions → reps value/reward of alt decisions
OFC neuron firing rate reflects the relative value of the choices

What are dopamine neurons?
D signals rep reward predict error
After learning → respond to reward predictor
Diagram uses ratterplots
