PSYC513 - Lecture 2 Notes

PSYC513: Biological and Cognitive Psychology

• Contact Information:
  • Email: matt.roser@plymouth.ac.uk
  • Office: PSQ B207
  • Office Hours: Tuesday 10-11am, Thursday 10-11am
  • Check-in Code: XX-XX-XX
  • Instructor: Dr. Matt Roser

Lecture 2: Neural Basis of Reward, Learning, Memory, and Drug Action

• Recommended Readings:
  • Banich, M., and Compton, R. Cognitive Neuroscience.
    • Chp 1: Intro to nervous system.
    • Chp 9: Learning and memory.
  • Carlson, N. Physiology of behaviour.
    • Chps 2-4: Structure of cells, Structure of nervous system, Psychopharmacology.
    • Chp 13: Learning and memory
  • Baars & Gage. Fundamentals of cognitive neuroscience: a beginner's guide.
    • Chp 2: The brain
    • Chp 7: Learning and memory

Major Thinkers

• Von Helmholtz and Neural Conduction (1849):
  • Measured the speed of axon potentials at $90 ft/sec$.
  • Refuted 'vitalism,' establishing the neural signal as physical (electrical) instead of a vital force.
  • Milestone in electrophysiology development.

Golgi, Cajal, and the Neuron Doctrine

• Santiago Ramón y Cajal:
  • Used Golgi’s method to create detailed neural assembly drawings.
  • Discovered the synapse and functional neuron polarity.
• Camillo Golgi:
  • Invented the silver nitrate staining method for neurons (Golgi method).
• The Neuron Doctrine Debate:
  • Cajal: The nervous system is composed of individual units that interact.
  • Golgi: The nervous system is a continuous mass.
  • Cajal’s view was proven correct.
• Both Cajal and Golgi shared the Nobel Prize in 1906.
• Gap junctions (electrical connections) provide some support for Golgi’s ideas.

Donald Hebb: The Organization of Behaviour (1949)

• Pioneered a comprehensive theory on how brain activity produces complex psychological phenomena (perceptions, emotions, thoughts, memories).
• Offered an alternative to behaviorism.
• Proposed that active cells form ‘assemblies’ as cognition elements.
• Sequences of assemblies (synfire chains) explain the perception of temporal patterns.

Neurons

• Cajal defined neurons as the nervous system's basic, distinct units.
• Cell Body (Soma):
  • Nucleus: Contains DNA for protein production.
  • Mitochondria: Produce adenosine triphosphate (ATP) for energy.
  • Also contains structures for protein synthesis and neurotransmitter transport.
• Processes:
  • Dendrites.
  • Axon (with myelin sheath: lipid and protein).
  • Terminal buttons.

Supporting Cells - Neuroglia

• Glia Cells:
  • Make up half of the brain’s volume in the central nervous system (CNS).
  • Astrocytes: Provide nutrients, structural support, clean-up, and chemical protection for neurons.
  • Oligodendrocytes: Form processes that produce the myelin sheath in the CNS.
• Peripheral Nervous System (PNS):
  • Schwann cells: Wrap neurons and support the regeneration of damaged axons.

Signalling and Information Flow in Neurons

• Diagram depicting the structure of a neuron, including:
  • Synapses on soma and dendrites.
  • Cell body.
  • Myelin sheath.
  • Axon.
  • Terminal button.

How Neurons Work: Membrane Potential and Ion Exchange

• Transmission is unidirectional: from dendrites to terminals.
• Dendrites and soma receive input, altering the neuron’s electrical charge (potential).
• Resting Potential: Charge across the neural membrane at rest (polarized at $-70mV$).
• Charge is maintained by ion concentration differences, diffusive and electrostatic pressures, and a mechanical process.
• Sodium ions ($Na^+$) must be kept at a greater concentration outside the cell due to low permeability and the sodium-potassium pump.
• The sodium-potassium pump exchanges $3 Na^+$ for $2 K^+$ ions.

Depolarization and the Action Potential

• Excitation (synaptic) from other neurons raises the membrane resting potential.
• When depolarization reaches a threshold, an action potential is triggered.
  • Voltage-dependent $Na^+$ channels open, allowing $Na^+$ to enter, increasing potential.
  • Voltage-dependent $K^+$ channels open.
  • $Na^+$ channels close.
  • $K^+$ leaves the cell, returning potential to normal.
  • The $Na^+/K^+$ pump restores concentrations over time.
• Depolarization occurs locally and spreads down the axon in an all-or-none fashion.

The Synapse

• Terminals lie adjacent to dendrites, soma, axons, or other terminals.
• Pre- and post-synaptic membranes are separated by the synaptic cleft.
• Synaptic vesicles contain neurotransmitter molecules.
• Action potential (AP) triggers neurotransmitter release into the cleft.

The Synapse – Neural Transmission

• Neurotransmitter binds to a receptor, opening ion channels and altering the postsynaptic membrane's polarization.
• Transmitter effects depend on the opened ion channel.
  • $Na^+$ channels: depolarization = Excitatory Post-Synaptic Potential (EPSP).
  • $K^+$ channels: hyperpolarization = Inhibitory Post-Synaptic Potential (IPSP).

Excitatory and Inhibitory Post-Synaptic Potentials

• Membrane potential is measured with an intracellular microelectrode.
• $Na^+$ channels open → depolarization → EPSP.
  • This moves the potential closer to the firing threshold.
• $K^+$ channels open → hyperpolarization → IPSP.
  • This moves the potential away from the threshold.

Neural Processing

• Ion channels open transiently; PSPs decay over time.
• Several EPSPs are needed for depolarization to reach threshold.
  • This allows summation of inputs from multiple neurons.
    • Temporal Summation: PSPs in close succession overlap and add.
    • Spatial Summation: Simultaneous PSPs at different locations (e.g., dendrites) add together.
  • EPSPs and IPSPs can cancel each other out.

Neurotransmitters

• Two primary transmitters: Glutamate and GABA (Gamma-Aminobutyric Acid).
  • Excitatory and inhibitory effects, respectively.
• Many (dozens) of modulatory transmitters.
• The effect on the post-synaptic neuron is determined by receptors present, neuron state, and other transmitter substances.
  • This allows complex neural processing modulation.
  • Nedergaard et al (2002). Beyond the role of glutamate as a neurotransmitter. Nature Reviews Neuroscience 3, 748-755

Neurotransmitters: Glutamate

• The brain’s most common excitatory transmitter.
• Increases the membrane potential of the postsynaptic cell, bringing it closer to the firing threshold.
• An amino acid produced by the neuron’s metabolism.
• Activates receptors named for drugs that affect them (e.g., NMDA, AMPA).
• AMPA receptor controls a $Na^+$ gate à EPSP.

Neurotransmitters: Glutamate - NMDA Receptor

• NMDA receptor controls $Na^+$ and $Ca^{2+}$ gates.
• $Ca^{2+}$ is involved in changes to AMPA receptors, producing Long Term Potentiation (LTP).
• NMDA receptor is blocked by $Mg^+$ ion.
  • Removed when the membrane is depolarized (by AMPA $Na^+$ channel).
• Other binding sites ($Zn^{2+}$) modulate NMDA receptor activity.

Neurotransmitters: GABA (Gamma-AminoButyric Acid)

• The brain’s most common inhibitory transmitter.
• Decreases the membrane potential of the postsynaptic cell, moving it further from the firing threshold (IPSP).
• Prevents excessive excitation.
• Inhibitory interneurons increase nervous system flexibility (e.g., suppress info, enhance contrast).
• Many receptor sites, allowing drug action.

Neurotransmitters: Dopamine

• Dopaminergic projections from substantia nigra (SN) and ventral tegmental area (VTA) modulate activity in striate, limbic, and cortical areas.
• SN modulates input areas of the basal ganglia (involved in action).
  • Degeneration causes Parkinson’s disease.
  • Treated with L-DOPA, a dopamine precursor.
• VTA is involved in reward and learning, or changing behavior to unexpected or highly salient stimuli.
  • Stimuli linked to VTA activation, especially the nucleus accumbens, are perceived as exciting or rewarding (mesolimbic system).
• Dopamine's effect can be excitatory, inhibitory, or modulatory (long-lasting effects) depending on the receptor.

Drugs and the Brain

• Drugs can be administered in various ways, with different time courses.
• They have effects once they reach the brain.
• The blood-brain barrier regulates the brain’s chemical environment.
  • Molecules, like drugs, must be transported across this barrier.

Drug Action

• Drugs typically affect processes in the synapses.
• Agonists: Facilitate post-synaptic effects.
• Antagonists: Inhibit post-synaptic effects.

Drug Action Mechanisms

• A diagram illustrates various mechanisms of drug action, including:
  • Drug serves as precursor (AGO, e.g., L-DOPA - dopamine).
  • Drug prevents neurotransmitter storage in vesicles (ANT, e.g., reserpine - monoamines).
  • Drug stimulates neurotransmitter release (AGO, e.g., black widow spider venom - ACh).
  • Drug inactivates synthetic enzyme, inhibits synthesis of neurotransmitter (ANT, e.g., PCPA - serotonin).
  • Drug inhibits neurotransmitter release (ANT, e.g., botulinum toxin - ACh).
  • Drug stimulates postsynaptic receptors (AGO, e.g., nicotine, muscarine - ACh).
  • Drug blocks postsynaptic receptors (ANT, e.g., curare, atropine - ACh).
  • Drug stimulates autoreceptors; inhibits synthesis/release of neurotransmitter (ANT, e.g., apomorphine - dopamine).
  • Drug blocks autoreceptors; increases synthesis/release of neurotransmitter (AGO, e.g., idazoxan - norepinephrine).
  • Drug blocks reuptake (AGO, e.g., cocaine - dopamine).
  • Drug inactivates acetylcholinesterase (AGO, e.g., physostigmine - ACh).

Examples of Drug Action

• Cocaine: A catecholamine agonist that blocks dopamine and norepinephrine reuptake.
• Benzodiazepines (e.g., Valium): GABA agonist; binds to a GABA receptor site, aiding GABA molecule binding, causing post-synaptic hyperpolarization and inhibitory effects.
• Alcohol: GABA agonist and NMDA (glutamate) receptor antagonist; also increases dopamine release (reward) and interferes with learning and memory.

Addiction

• Drugs are addictive because drug-taking behavior is reinforced.
  • Positive Reinforcement: Presentation of an appetitive stimulus (e.g., a heroin ‘rush’) associated with a behavior.
  • Negative Reinforcement: Removal of an aversive stimulus (e.g., anxiety) associated with a behavior.
• Reinforcement from natural stimuli or drugs is linked to dopamine release in the nucleus accumbens (part of the mesolimbic dopaminergic system).

Addiction: Temporal Proximity

• Temporal proximity (of drug and behavior) is important.
• Heroin is more addictive than morphine because it crosses the blood-brain barrier faster, making it a more effective reinforcer.
• Animal self-stimulation is a better reinforcer than food unless food is delivered immediately.

What are Learning and Memory?

• Learning: Acquisition of information.
  • Encoding (sensory, representation, associative, motor).
• Memory: Retention of information.
• NOT a filing system!
• Learning changes the brain, thus altering perception, performance, thought, and planning.

Learning: Classical Conditioning

• Involves associating two stimuli and an automatic response.
• An Unconditional Stimulus (US) causes an Unconditional Response (UR).
• If the US is paired with a neutral stimulus, the neutral stimulus may elicit the Conditioned Response (CR).
• Ivan Pavlov and dog experiment.

Learning: Instrumental / Operant Conditioning

• Involves associating a learned response and a stimulus.
• Reinforcement, through presenting an appetitive stimulus and successive refinements of a complex behavior, strengthens associations between stimulus and response (the behavior).
• B.F. Skinner and rat experiments.

Learning and the Brain

• Hebb’s Rule: A synapse repeatedly active when the postsynaptic neuron is firing will be strengthened.
• Classical Conditioning:
  • US (puff) → UR via strong synapse.
  • Pairing tone with US strengthens weak synapse.
  • Tone then becomes a conditioned stimulus (CS).

Learning and the Brain: Instrumental / Operant Conditioning

• Hebb’s Rule: A synapse repeatedly active when the postsynaptic neuron is firing will be strengthened.
• Instrumental / Operant conditioning
  • Reinforcement system strengthens an association between a perception (lever) and a behaviour (pressing)
• Basal ganglia integrates perception and action planning.
  • Destruction leads to failure of instrumental conditioning

Reinforcement

• Many brain areas and neurotransmitter systems are associated with reward, particularly the ventral tegmentum (VT) and dopamine.
• Electrical microstimulation of VT (lower midbrain) can have reinforcing effects on behaviors, similar to natural reinforcers (food/sex).
• VT projects to the nucleus accumbens, prefrontal cortex, hippocampus, and amygdala (limbic system) - the Mesolimbic Dopaminergic System.
• Extracellular dopamine concentration (rats) and fMRI activation (humans) increased in Nucleus Accumbens: VT stimulation – rats; money reward - humans.

Reinforcement: Dopamine Neuron Response

• Dopamine (DA) neurons respond to reward when a monkey is learning a task.
• Initial reward is unexpected because performance is unreliable.
• Once the monkey reliably performs the task, the reward becomes expected, and the DA neuron stops responding to the reward (Schulz et al 1993).

Reinforcement: Mesolimbic Dopamine System

• Unexpected reward (novel food) → strong positive dopamine signal.
• Signal declines with repeated presentation and learning.
• Eventually, presentation of a predicted reward → no dopamine signal.
• Dopamine in Nucleus Accumbens (NA) surges in response to the predictive stimulus.
• Omission of predicted reward leads to suppression of the dopamine signal.
• Midbrain dopaminergic neurons' response represents a learning signal coding for errors in prediction of reward.
• Learning associations between two non-reinforcing stimuli is also linked to enhanced dopamine release in NA, which also receives excitatory input from the medial prefrontal cortex.
• The mesolimbic dopamine system modulates associative learning in general, not only reinforcement-based learning.

Reinforcement: Dopamine and Learning

• Dopamine antagonists (reduce DA activation) block reinforcement learning.
• Dopamine modulates Long-Term Potentiation (LTP).
• D1 receptor activation enhances the activity of neurons receiving strong excitatory input and reduces the activity of neurons receiving weak inputs.
• Dopaminergic reward systems are powerful modulators of learning.
• They modulate how learning is instantiated in the brain as memory.

Memory in the Brain

• Circa 1900, Cajal proposed that synaptic connections between neurons mediating behavior are modified by learning.
• These modifications can persist and serve as memory.
• Synaptic connection strength can be increased by sensitization (following prolonged stimulation) and classical conditioning, and reduced by habituation.
• The storage of non-declarative memory is embedded in the neural circuit that produces the behavior – unconscious memories such as skills.
• Does declarative memory in mammals also involve synaptic change? - Memories that can be consciously recalled, such as facts and events.

Hippocampus and Memory Consolidation

• Amnesia: A deficit in memory resulting from brain damage (hypoxia, surgery), specific to memory subcomponents.
  • Anterograde Amnesia: Inability to form new long-term memories following insult/injury.
  • Retrograde Amnesia: Inability to recall memories preceding insult.
• Medial Temporal Lobe & Hippocampus.
• ECT: electroconvulsive therapy.

Hippocampus and Memory Consolidation: Case H.M.

• Case H.M.: Surgery for epilepsy involved bilateral resection of the medial temporal lobe (MT), including the hippocampus.
  • Normal working memory (digit span), perceptual learning, instrumental and classical conditioning.
  • Disrupted transfer from short-term to long-term memory → dense anterograde amnesia.
  • The hippocampus is not necessary for short-term memory.
  • Previous declarative memory was intact, thus the hippocampus is not the repository of long-term memory.
  • The hippocampus is critical for the consolidation of new memories.

Cellular Basis of Long-Term Learning: Long-Term Potentiation (LTP)

• Principle: If a weak and a strong input act on a neuron simultaneously, the weak synapse becomes stronger (Hebbian learning).
• When the same weak input is given again, the response of the target cell increases (LTP=memory).
• For LTP to be induced:
  • The postsynaptic cell must be depolarized (by strong input).
  • The postsynaptic cell must receive additional input (weak input).
• In 1966, long-term synaptic potentiation (LTP) (strengthening) was observed in the hippocampus (first paper in 1973).
• Use-dependent strengthening of synaptic connections.

Long-Term Potentiation (LTP)

• Hebb's Law: 'cells that fire together wire together'
• Active synapse & postsynaptic neuron → strengthening occurs

Long-Term Potentiation (LTP): Synaptic Mechanisms

• Excitatory neurotransmitter: Glutamate
• NMDA receptors
  • Contain transmembrane channel for $Ca^{2+}$
  • Double gated – transmitter & voltage
• Glutamate binds to NMDA
• Other receptors depolarize the cell, removing $Mg^+$ block
  • Need presynaptic + postsynaptic activity
• $Ca^{2+}$ channel opens, $Ca^{2+}$ activates enzyme
• Triggers
  • insertion of more AMPA receptors into the membrane
  • retrograde messenger (NO) leads to increased presynaptic glutamate release
  • Stronger synaptic response, bigger EPSPs

Long-Term Depression

• Low-frequency stimulation of a synapse or firing of two inputs out of phase results in its weakening.
• Inputs that do not contribute to postsynaptic firing are weakened.
• Thus, learning is instantiated in the brain as memory.

Summary

• Neurons work as a network of units to process information.
• Synaptic transmission involves chemical signals.
• Drugs affect synaptic transmission in many ways.
• There are many networks in the brain that rely on different neurotransmitters.
• Reinforcement can increase the likelihood of a behavior reoccurring.
• Learned behaviors, experience and memory are instantiated in the brain as patterns of synaptic strength.