Lecture 2: Neural Basis of Reward, Learning, Memory, and Drug Action
Recommended Reading:
Banich, M., and Compton, R. Cognitive Neuroscience.
Chp 1. Intro to nervous system.
Chp 9. Learning and memory.
Carlson, N. Physiology of behaviour.
Chps 2-4. Structure of cells, Structure of nervous system, Psychopharmacology.
Chp 13. Learning and memory
Baars & Gage. Fundamentals of cognitive neuroscience: a beginner's guide.
Chp 2. The brain
Chp 7. Learning and memory
Major Thinkers
Von Helmholtz and neural conduction – 1849
Measured the speed of axon potentials (90ft/sec).
Refuted the hypothesis of 'vitalism' – the neural signal was physical (electrical) not a vital force of nature specific to particular senses.
Milestone in the development of electrophysiology.
Golgi, Cajal, and the Neuron Doctrine
Santiago Ramón y Cajal
Used Golgi’s method to produce detailed drawings of neural assemblies.
Discovered the synapse and functional polarity of neurons – direction signals travel.
Camillo Golgi
Invented the method of staining neurons with silver nitrate, the ‘Golgi’ method.
Disagreement
They disagreed over whether the nervous system was composed of individual units that interact with their neighbours (Cajal) or was a continuous mass (Golgi).
Cajal was proved correct, but both shared the Nobel Prize in 1906.
Gap junctions (electrical connections) support Golgi’s ideas to some extent.
Donald Hebb
The Organization of Behaviour (1949)
The first comprehensive theory of how complex psychological phenomena, perceptions, emotions, thoughts, and memories, might be produced by brain activity.
An alternative to the dominant paradigm in psychology - behaviorism.
Stated simple rules on how active cells could form ‘assemblies’ to form the elements of cognition.
Sequences of assemblies (‘synfire chains’) could explain the perception of temporal patterns, i.e., those that unfold in time.
Neurons
Cajal - Neurons are the basic, distinct units of the nervous system.
Cell body or Soma
Nucleus: contains DNA which codes for the production of proteins which serve many functions within the cell.
Mitochondria: produce adenosine phosphate (ATP) which is used for energy.
Other structures involved in protein synthesis and transport (e.g., of neurotransmitters).
Excitatory and Inhibitory Post-Synaptic Potentials
Membrane potential measured with an inserted (intracellular) microelectrode.
Na+ channels open à depolarization à EPSP.
This moves the potential closer to the firing threshold.
K+ channels open à hyperpolarization à IPSP.
This moves the potential away from the threshold.
Neural Processing
Ion channels open transiently - PSPs decay over time.
Several EPSPs are necessary for depolarization to reach the threshold.
This allows inputs from many neurons to be added together (summated).
Temporal summation - PSPs in close succession will overlap and add.
Spatial summation - simultaneous PSPs at different locations (e.g., dendrites) will add.
EPSPs and IPSPs can cancel each other.
Neurotransmitters
Two primary transmitters - Glutamate and GABA (Excitatory and Inhibitory effects).
Many (dozens) of modulatory transmitters.
The effect on the post-synaptic neuron is determined by the receptors present, the state of the neuron, and the presence of other transmitter substances.
This allows the complex modulation of neural processing.
Nedergaard et al (2002). Beyond the role of glutamate as a neurotransmitter. Nature Reviews Neuroscience 3, 748-755.
Neurotransmitters: Glutamate
The brain’s most common excitatory transmitter.
Increases the membrane potential of the postsynaptic cell (Brings the cell closer to the firing threshold).
It is an amino acid produced by the neuron’s metabolism.
Activates several types of receptors - named for the drugs which affect them (e.g., NMDA, AMPA).
AMPA receptor controls a Na+ gate à EPSP.
Neurotransmitters: Glutamate - NMDA receptor
NMDA receptor - controls Na+ and Ca2+ gates.
Ca2+ is involved in changes to AMPA receptors, producing Long Term Potentiation (LTP).
NMDA receptor is blocked by Mg+ ion.
This is removed when the membrane is depolarized (by AMPA Na+ channel).
Other binding sites (Zn2+) have modulatory effects on NMDA receptor.
Neurotransmitters: GABA (Gamma-AminoButyric Acid)
The brain’s most common inhibitory transmitter.
Decreases the membrane potential of the postsynaptic cell (Takes it further from the firing threshold - IPSP).
Prevents excessive excitation.
Inhibitory interneurons increase the flexibility of the nervous system (e.g., suppress info, enhance contrast).
Many receptor sites, allowing drug action.
Neurotransmitters: Dopamine
Dopaminergic projections from substantia nigra (SN) and ventral tegmental area (VTA) modulate activity in striate, limbic, and cortical areas.
SN modulates input areas of the basal ganglia (involved in action).
Degeneration causes Parkinson’s disease.
Treated with L-DOPA, a precursor of dopamine.
VTA is involved in reward and learning or, more generally, in changing behavior to unexpected or highly salient stimuli.
Stimuli associated with VTA activation, particularly the nucleus accumbens, are perceived as exciting or rewarding (mesolimbic system).
The effect of dopamine can be excitatory, inhibitory, or modulatory (long-lasting effects) depending on the receptor.
Drugs and the Brain
Drugs can be administered several ways, with different time courses.
They have effects once they reach the brain.
The blood-brain barrier aids the regulation of the brain’s chemical environment.
Molecules, such as drugs, must be transported across this barrier.
Drug Action
Drugs typically affect processes in the synapses.
Agonists - facilitate post-synaptic effects.
Antagonists - inhibit post-synaptic effects.
This occurs in many ways……
Drug Action (Detailed List)
Drug serves as a precursor (AGO, e.g., L-DOPA-dopamine).
Drug prevents the storage of NT in vesicles (ANT, e.g., reserpine-monoamines).
Drug stimulates release of NT (AGO, e.g., black widow spider venom-ACh).
Drug inactivates synthetic enzyme; inhibits synthesis of NT (ANT, e.g., PCPA-serotonin).
Drug inhibits release of NT (ANT, e.g., botulinum toxin-ACh).
Drug stimulates postsynaptic receptors (AGO, e.g., nicotine, muscarine-ACh).
Drug blocks postsynaptic receptors (ANT, e.g., curare, atropine-ACh).
Drug stimulates autoreceptors; inhibits synthesis/release of NT (ANT, e.g., apomorphine-dopamine).
Drug blocks autoreceptors; increases synthesis/release of NT (AGO, e.g., idazoxan-norepinephrine).
Drug blocks reuptake (AGO, e.g., cocaine-dopamine).
Drug inactivates acetylcholinesterase (AGO, e.g., physostigmine-ACh).
Examples
Cocaine - a catecholamine agonist, blocks reuptake of dopamine and norepinephrine.
Benzodiazepines (e.g., Valium) - GABA agonist, binds to one site on the GABA receptor and aids binding of GABA molecules à post-synaptic hyperpolarization and subsequent inhibitory effects.
Drugs are addictive because drug-taking behavior is reinforced.
Positive reinforcement - the presentation of an appetitive stimulus (e.g., a heroin ‘rush’) in association with a behavior.
Negative reinforcement - the removal of an aversive stimulus (e.g., anxiety) in association with a behavior.
Reinforcement, either from natural stimuli or from drugs, is linked to the release of dopamine in the nucleus accumbens - part of the mesolimbic dopaminergic system.
Addiction (cont.)
Temporal proximity (of drug and behavior) is important.
Both heroin and morphine are converted to dopamine in the brain, but heroin is more addictive.
Heroin crosses the blood-brain barrier faster than morphine, so is a more effective reinforcer.
Animal self-stimulation is a better reinforcer than food, unless food is delivered immediately.
Learning changes the brain and hence the way we perceive, perform, think, and plan.
Learning: Classical Conditioning
Involves the association between two stimuli and an automatic response.
An Unconditional Stimulus (US) causes an Unconditional Response (UR).
If the US is paired with a neutral stimulus, the neutral stimulus can come to elicit the, now conditioned, response (CR).
Ivan Pavlov and dog.
Learning: Instrumental / Operant Conditioning
Involves an association between a learned response and a stimulus.
Reinforcement, through the presentation of an appetitive stimulus, and successive refinements of a complex behavior leads to the strengthening of associations between a stimulus and a response (the behavior).
BF Skinner and rat.
Learning and the Brain
Hebb’s Rule - a synapse that is repeatedly active when the postsynaptic neuron is firing will become strengthened.
Classical conditioning
US (puff) à UR via strong synapse.
Pairing of tone with US strengthens the weak synapse.
Leading to tone becoming a conditioned stimulus CS.
Learning and the Brain
Hebb’s Rule - a synapse that is repeatedly active when the postsynaptic neuron is firing will become strengthened.
Instrumental / Operant conditioning
The reinforcement system strengthens an association between a perception (lever) and a behavior (pressing).
The basal ganglia integrate perception and action planning.
Destruction leads to failure of instrumental conditioning.
Reinforcement
Many brain areas and neurotransmitter systems are identified with reward, the ventral tegmentum (VT) and dopamine in particular.
Electrical microstimulation of VT (lower midbrain) can have reinforcing effects on behaviors, similar to natural reinforcers (food/sex).
VT projects to the nucleus accumbens, prefrontal cortex, hippocampus, and the amygdala (limbic system) - the Mesolimbic Dopaminergic System.
Extracellular dopamine concentration (rats) and fMRI activation (humans) increased in Nucleus Accumbens.
VT stimulation – rats
Money reward - humans
Reinforcement (cont.)
Dopamine (DA) neuron responds to the reward during the period when the monkey is learning a task.
In this case, its performance is not yet reliable, and so the reward is ‘unexpected’.
Once the monkey can reliably perform the task, the reward becomes ‘expected,’ and the DA neuron stops responding to the reward. (Schulz et al 1993).
Reinforcement (cont.)
Unexpected reward (novel food) à strong positive dopamine signal.
Declines with repeated presentation and learning.
Eventually, the presentation of a predicted reward à no dopamine signal.
But dopamine in Nucleus Accumbens (NA) surges in response to the predictive stimulus.
Omission of a predicted reward leads to the suppression of the dopamine signal.
Response of midbrain dopaminergic neurons represents a learning signal that codes for errors in the prediction of reward.
Learning associations between two non-reinforcing stimuli is also linked to enhanced dopamine release in NA, which also receives excitatory input from the medial prefrontal cortex.
The mesolimbic dopamine system is involved in the modulation of associative learning in general, not only that involving reinforcement.
Reinforcement (cont.)
Dopamine antagonists (reduce DA activation) block reinforcement learning.
Dopamine modulates Long-Term Potentiation (we will return to this later).
D1 receptor activation enhances the activity of neurons receiving strong excitatory input from other sources and reduces the activity of neurons receiving weak inputs.
Dopaminergic reward systems are a powerful modulator of learning.
They modulate how learning is instantiated in the brain as Memory.
Memory in the Brain
Circa 1900 Cajal proposed that synaptic connections between neurons mediating behavior are not static but become modified by learning.
These modifications can persist and serve as memory.
The strength of synaptic connections can be increased by sensitization (following prolonged stimulation) and classical conditioning and be reduced by habituation.
Thus, the storage of non-declarative memory is embedded in the neural circuit that produces the behavior – unconscious memories such as skills.
Does declarative memory in mammals also involve synaptic change?
Memories which can be consciously recalled such as facts and events.
Hippocampus and Memory Consolidation
Amnesia: a deficit in memory.
Result of brain damage (hypoxia, surgery) and can be specific to memory subcomponents.
Anterograde amnesia – inability to form new long-term memories following insult/injury.
Retrograde amnesia – inability to recall memories preceding insult.
Medial Temporal Lobe & Hippocampus
ECT: electroconvulsive therapy.
Hippocampus and Memory Consolidation: H.M.
H. M., an Unforgettable Amnesiac, Dies at 82; The New York Times
Case H.M.
Surgery for epilepsy: Bilateral resection of the medial temporal lobe (MT) including the hippocampus.
Normal working memory (digit span), perceptual learning, instrumental, and classical conditioning.
Disrupted transfer from short-term to long-term memory à dense anterograde amnesia.
Hippocampus not necessary for short-term memory.
Previous declarative memory intact, thus the hippocampus is not the repository of long-term memory.
Hippocampus is critical for the consolidation of new memories.
Cellular Basis of all Long-Term Learning
Use-dependent strengthening of synaptic connections: long-term-potentiation (LTP).
Principle: If a weak and a strong input act on a neuron at the same time, the weak synapse becomes stronger (Hebbian learning).
This means, when the same weak input is given again, the response of the target cell increases (LTP=memory).
For LTP to be induced, two things must be in place:
The postsynaptic cell must be depolarized (by strong input).
The postsynaptic cell must receive additional input (weak input).
In 1966, long-term synaptic potentiation (LTP) (strengthening) was observed in the hippocampus (first paper 1973).
Long-Term Potentiation (LTP)
Hebb's law: 'cells that fire together wire together'
Active synapse & postsynaptic neuron → strengthening
Diagram depicting the stimulation of an axon that forms a synapse with a neuron, depolarization of the cell, and strengthening of the synapse.
Long-Term Potentiation (LTP): Synaptic Mechanisms
Excitatory neurotransmitter - Glutamate
NMDA receptors
Contain transmembrane channel for Ca2+.
Double gated – transmitter & voltage.
Glutamate binds to NMDA.
Other receptors depolarize the cell, removing Mg+ block.
Need pre-synaptic + post-synaptic activity.
Ca2+ channel opens, Ca2+ activates enzyme.
Triggers:
Insertion of more AMPA receptors into the membrane.
Retrograde messenger (NO) leads to increased presynaptic glutamate release.
Stronger synaptic response, bigger EPSPs.
Long-Term Depression
Low-frequency stimulation of a synapse, or firing of two inputs out of phase, results in its weakening.
Inputs that do not contribute to postsynaptic firing are weakened.
Thus, learning is instantiated in the brain - as memory.
Summary
Neurons work as a network of units to process information.
Synaptic transmission involves chemical signals.
Drugs affect synaptic transmission in many ways.
There are many networks in the brain that rely on different neurotransmitters.
Reinforcement can increase the likelihood of a behavior reoccurring.
Learned behaviors, experience, and memory are instantiated in the brain as patterns of synaptic strength.