Biopsychology Lecture Notes: Decision Making, Reward, and Plasticity

Choice, Reward, and Decision Making

  • Event context (Week-long neuroscience fair talks): Speakers from the Poe Fly Lab, Memory and Eye Movements Lab, Fullstack Affective Neuroscience Lab, and others; food provided. Date/time: Wed, Sep 17, 2025, 05:00–06:30 pm; location: JBHT 0144.
  • Schedule highlights: This week topics include Choice & Plasticity (Tue) and Learning & Memory (Thu); Next week includes Review 1 (Tue) and Exam 1 (Thu). Required readings: week4spike, week4neuro2064. No assignment; timeline mentioned.
  • Quick theme: Applying neuroscience methods to study how you “choose” and how brain plasticity underpins learning from experience. Mention of designer drug / designer receptor (likely a nod to modern neuroscience tools such as DREADDs).
  • Video resource: vid: https://youtu.be/de_b7k9kQp0?t=2m10s
  • Quiz and practice: Quiz (graded for completion) on “choice” at https://fullstackneuro.io/biopsyc/ or via QR code.

Core concepts: What drives human choice

  • Decision variables often considered in models of choice:
    • Reward magnitude (amount of gain or loss)
    • Probability of receiving the reward
    • Time of reward (now vs. later)
    • Risk (potential gain and potential loss)
    • These together enable deconstruction of human choice behavior. extChoice=f(Rm,P,T,extrisk)ext{Choice} = f(R_m, P, T, ext{risk})
  • Context slide emphasizes a framework for linking subjective value to neural signals.

Neuroanatomy: brain regions implicated in choice and cognition

  • Frontal Lobe
    • Motor control (motor cortex)
    • Cognitive activities (planning, decision making, problem solving)
    • Language (Broca’s area)
  • Parietal Lobe
    • Sensation (somatosensory cortex: touch, pressure, temperature)
    • Fine sensation (texture, size, shape)
    • Spatial awareness
  • Temporal Lobe
    • Hearing (auditory cortex)
    • Memory (hippocampus)
    • Language (Wernicke’s area)
  • Occipital Lobe
    • Vision
  • Neuroanatomy: Midsagittal plane (location of major structures in a sagittal view)

The Triune brain and the dopaminergic reward system

  • Paul MacLean’s Triune Brain framework:
    • Neomammalian (human cortex) – symbolic processing
    • Paleomammalian (limbic system) – emotions and social processing
    • Reptilian (brainstem) – basic survival functions; the 4 F's: Feeding, Fighting, Fleeing, Reproduction (context note: “N, P, R” in slide shorthand)
  • Mesolimbic dopamine system:
    • Ventral tegmental area (VTA) produces dopamine
    • Dopamine projection targets include nucleus accumbens (NAcc) and medial prefrontal cortex (MPFC)
  • Core idea: dopamine signaling links anticipated rewards to motivational drive and action selection.

Imaging methods and experimental tasks

  • Functional Magnetic Resonance Imaging (FMRI)
    • Temporal resolution ~2 seconds
    • Spatial resolution ~3 mm
    • Setup: projector, computer, scanner, response box, etc.
  • Monetary Incentive Delay (MID) task (Knutson et al., 2000):
    • Conditions during trials: Anticipation of gain (+$5), Anticipation of loss (-$5), Anticipation of no money ($0)
    • Trial structure: cue → delay → target → feedback
    • Purpose: dissociate neural signals during anticipation of gain vs. loss vs. neutral outcomes
  • Outcome summary from MID research:
    • Gain anticipation activates NAcc (nucleus accumbens) – the same region implicated in rodent self-stimulation.
    • Loss anticipation activates Anterior Insula (AIns).
    • MPFC (medial prefrontal cortex) is involved in coding reward probability and value signals during anticipation (as seen in condition-specific responses).
    • The endowment effect and related valuation processes link to AIns activity during loss/gain framing.

Key studies and findings in reward processing

  • Knutson et al., 2000 (MID task):
    • Gain anticipation → NAcc activation
    • Loss anticipation → Anterior Insula activation
    • Data illustrate how different brain regions encode aspects of motivational value before outcomes.
  • Knutson et al., 2005 (Probability and MPFC):
    • Probability of reward and magnitude interact to shape MPFC activity during anticipation
    • Example conditions: +$5 with 80% probability, +$5 with 50% probability, +$5 with 20% probability
    • MPFC shows coding related to the expected value across varying probabilities of gain
    • Representative data point: A = 14, PRB (probability) cue → anticipation response in MPFC
  • Knutson et al., 2005 (quantitative symbols):
    • Under +$5/80%, +$5/50%, +$5/20% conditions, MPFC response tracks the probabilistic value of the anticipated reward
  • Knutson et al., 2000/2005 synthesis takeaways:
    • Gain magnitude is coded in NAcc
    • Loss magnitude is coded in AIns
    • Probability of obtaining reward is coded in MPFC
    • Time (now vs. later) involves LPFC (for delayed rewards and executive control)

Endowment effect and anterior insula

  • Endowment effect: owning an object increases its subjective value; “it hurts to lose something you love.”
  • Data example: endowment of a favorite Razorbacks cup used to demonstrate the effect with personal data.
  • Anterior Insula (AIns) activity correlates with larger endowment effects (i.e., stronger loss aversion when owning an item).
  • Additional figure (Knutson et al., 2008) supports the link between AIns activation and endowment-related valuation changes.

Valuing gains, losses, and decision signals across networks

  • Gaining vs losing reward:
    • Gain magnitude coding in NAcc
    • Loss coding in AIns
    • Probability of getting reward coded in MPFC
    • Time of reward (now vs. later) coded in LPFC for delayed decisions
  • Structural connectivity and gambling behavior (AIns–NAcc):
    • Stronger white-matter connections between AIns and NAcc associated with not gambling as a behavior pattern (Knutson et al. lineages and follow-ups, Leong et al.).
    • Note of reported correlations: c = -0.40, c’ = -0.24, a = -0.35, b = 0.46** (illustrative path coefficients for a mediation-style interpretation; p-values indicated by * / **).
  • Practical question raised: How can real-time brain signals be used?
    • Every second, one could in principle gauge how much someone likes/dislikes a stimulus or experiences loss aversion
    • Potential applications: marketing, product design, behavioral interventions, and neuromarketing research (with ethical considerations).

Applied examples: Predicting consumer choices

  • Predict buying (GODIVA chocolate; Knutson et al., 2005):
    • Task: product shown with price; subjects decide Yes/No while NAcc (and related regions) are measured
    • Data patterns show NAcc activity tracks anticipated reward value and can predict purchase choices
    • Example figures show sequential trials with fixations and choices; NAcc signal correlates with eventual decision
  • Predict buying: additional variables in datasets include fixations, choice timing (4 s windows), and subjective value estimates.
  • Predict not buying: similar MID-like patterns show Insular regions modulate decisions to refrain from purchases; data suggest insula activity relates to avoidance of costly or less desirable options.

Cross-domain demonstrations: music and streaming sales

  • Predict music sales (Berns & Moore, 2012):
    • Experimental paradigm: listen to songs; collect ratings (popularity, familiarity) and actual sales data (albums sold)
    • Finding: NAcc activation during listening correlates with subsequent sales performance (R ≈ 0.32)
  • Predict streaming/view trials (Tong et al., 2020):
    • Design: participants view trial blocks of videos, rate engagement/valence/arousal; measure NAcc activity during viewing
    • Result: NAcc activity correlates with view frequency and engagement metrics (approximate r ≈ 0.46, p < .01 in reported data)
  • Takeaway: reward-related neural signals can predict real-world consumer behaviors across products and media.

Time, delay, and future reward valuation

  • Time-based decision tasks:
    • Classic questions: Would you rather have $10 today or $15 in 30 days?
    • Neural coding patterns:
    • Immediate rewards vs. delayed rewards invoke different prefrontal circuits
    • Dorsolateral Prefrontal Cortex (DLPFC) and lateral prefrontal cortex (LPFC) implicated in computing long-term value and control in timed choices
  • Ballard et al. (2009) findings on brain regions during time-based value tasks:
    • Dorsolateral prefrontal cortex (DLPFC) and inferior frontal gyrus (IFG) involvement in temporal valuation and cognitive control
  • Summary of time/value coding:
    • Reward magnitude: NAcc
    • Loss magnitude: AIns
    • Probability: MPFC
    • Time/Delay: LPFC and DLPFC involvement in delayed rewards

Integrating time, risk, and value: Ballard and others

  • Time-focused valuation analyses (Ballard et al., 2009):
    • Brain coordinates show activity in dorsolateral and lateral prefrontal regions during timing and value integration
    • Equations/symbols from figures show intent to map temporal aspects to specific cortical loci (e.g., y = 55 in one schematic for dorsolateral PFC activation; general idea of spatial mapping of time/value signals)
  • In short: the brain integrates magnitude, probability, and delay across distributed circuits to compute net value and guide choice.

Risk and skew in real-world decisions

  • Real-world risk preferences:
    • Humans tend to like positively-skewed risks (e.g., lotteries, casino games) more than symmetric or negatively skewed risks
    • Traditional finance models often do not account for skewness in risk preference
    • Question: can neuroscience illuminate why people prefer asymmetric risks?
  • Kraus & Litzenberger (1976) reference to risk in finance as a backdrop for neuroeconomic inquiry

Risky gambling task and neural correlates

  • Gambling paradigm (Wu, Bossaerts, Knutson et al. context):
    • Conditions comparing Positive-skew vs Symmetric vs Negative-skew gambles and a No Gamble option
    • Measured outcomes include gamble rate and NAcc activation patterns over time (seconds)
  • Key findings:
    • Positive-skew gambles elicit stronger NAcc activity compared to symmetric/negative-skew gambles
    • Neural responses align with behavioral tendency to engage more with positive-skew risks
  • Graph interpretation notes:
    • Time-series of Signal Change in NAcc shows peak responses following presentation of skewed gambles
    • Statistical markers indicate significance for the positive-skew condition in relation to risk-taking behavior

Neuroanatomical and functional connections in gambling behavior

  • Anterior Insula–NAcc structural connectivity:
    • Structural connections between AIns and NAcc relate to gambling propensity
    • Stronger AIns–NAcc white-matter connectivity associates with not engaging in gambling
  • Regression-style coefficients (illustrative from slides):
    • c = -0.40, c’ = -0.24, a = -0.35, b = 0.46**
    • Interpretation: certain pathways moderate the relationship between affective signals (insula) and reward processing (NAcc) in predicting gambling behavior

Takeaway points: Integrated view of choice

  • Core coding in different regions:
    • Reward magnitude (gain) → NAcc
    • Loss magnitude → AIns
    • Reward probability → MPFC
    • Time/delay → LPFC
    • Risk processing involves interactions between NAcc and AIns, modulated by structural connectivity
  • The brain’s choice architecture is distributed and interdependent; simple one-region explanations are insufficient.
  • Structural connections (white matter) between key regions are an emerging frontier for linking brain networks to complex behavior (e.g., gambling risk-taking).
  • While powerful, these neural signals are only part of the story; context, modelling assumptions, and ethical considerations matter in applying neuroscience to real-world decisions.

Meta-analytic and critical takes: Recap and cautions

  • Recap: Recap assignment 3 discusses NeuroSynth as a platform for linking FMRI activity to keywords across thousands of studies; dynamic meta-analyses can search by term or brain area to map terms to brain activity and vice versa.
  • Week 3 readings (reverse inference) explain why we cannot infer a specific cognitive process solely from brain area activation; multiple processes can co-activate the same region.
  • Week 3 readings warn against neuro hype and discuss what neuroimaging actually measures (and when it should be used), with examples where neuroimaging may be unnecessary or misleading.

Plasticity: the brain’s capacity to change across scales

  • Core idea: Plasticity refers to brain changes due to experiences with the environment across spatial and temporal scales.
  • Types of plasticity:
    • Grey matter plasticity
    • White matter plasticity
    • Synaptic plasticity
  • Grey matter plasticity:
    • Involves growth and organization of neurons in cortical columns and layers
    • Key historical examples: monocular deprivation effects on primary visual cortex (Hubel & Wiesel, 1964); structural changes observed by Maguire et al. (2000); Draganski (2004)
  • White matter plasticity:
    • Changes in myelination, axon diameter, and number of axons within a tract to speed and synchronize communication between brain areas
    • White matter changes observed massively after birth; pruning during development (roughly ages 3–19, maybe until 25); ongoing debate about adult plasticity (Anguera et al., 2013; Sagi et al., 2012)
  • Synaptic plasticity:
    • Strengthening or creation of synapses between neurons (Hebbian-like principles: “cells that fire together wire together”)
    • Core mechanisms involve NMDA receptors as coincidence detectors, calcium influx, and downstream signaling that strengthens synapses

Action potentials and synaptic mechanisms: foundations recap

  • Action potential basics (review slides):
    • Glutamate binds to liganded Na+ channels → excitatory postsynaptic potential (EPSP)
    • EPSP leads to depolarization; reaches threshold to open voltage-gated Na+ and K+ channels, triggering an action potential
    • Classic scale: resting potential ≈ $V{rest} \approx -70\,\text{mV}$; threshold ≈ $V{th} \approx -55\,\text{mV}$
  • Presynaptic events:
    • Action potential arrives at axon terminal; voltage-gated Ca^{2+} channels open
    • Vesicles release neurotransmitter into synaptic cleft
    • Neurotransmitter binds to postsynaptic receptors to propagate the signal
  • NMDA receptor and synaptic plasticity:
    • NMDA receptors act as a coincidence detector: require both glutamate binding and postsynaptic depolarization (to relieve Mg^{2+} block) and Ca^{2+} influx
    • When Glutamate and Ca^{2+} coincide, NMDA receptors activate and trigger signaling cascades that induce synaptic changes
    • Result: LTP (Long-Term Potentiation) – increases the efficacy of future synaptic transmission; foundational mechanism for learning, memory, and addictions (to be expanded next week)
  • Mechanistic summary for LTP (condensed):
    • If Glutamate and Ca^{2+} are present together at the synapse → NMDA receptor activation → intracellular signaling → structural and functional synaptic strengthening → greater future postsynaptic responses to the same presynaptic input

Spatial scales: from nanometers to meters

  • Spatial scale sense:
    • Grey matter and white matter operate at micrometer-to-millimeter scales (roughly 10^-6 to 10^-3 meters)
    • Synaptic changes occur at nanometer scales (roughly 10^-9 meters)
  • Visualizing scale progression:
    • Classic scale demonstrations (e.g., scale models and animations) help contextualize how micro-level synaptic changes underpin macro-level cognitive functions
  • Practical thought: many brain changes underlying learning are multiplexed across these scales and over different timescales (development, adulthood, aging)

Quiz and practice resources

  • Quiz 2 (graded for completion) on plasticity: https://fullstackneuro.io/biopsyc or QR code provided in slides

LaTeX and key numerical details (quick reference)

  • Resting potential: Vrest ≈ −70 mVV_{rest} \,\approx \, -70\ \text{mV}
  • Action potential threshold: Vth ≈ −55 mVV_{th} \,\approx \, -55\ \text{mV}
  • MID task conditions (gain/loss/neutral): gains of +5.00+5.00, losses of −5.00-5.00, neutral +0.00+0.00; probabilities often set as P \in \{0.80, 0.50, 0.20} for different cue conditions
  • Reward magnitude coding locations: nucleus accumbens (NAcc) / gain; anterior insula (AIns) / loss; probability coding in MPFC; delayed reward coding in LPFC
  • Example correlation: Berns & Moore (2012) reported a correlation of approximately R=0.32R = 0.32 between NAcc activation and quantity of albums sold
  • Positive-skew gambling results: indicates greater NAcc activation for positive-skew gambles relative to other skew types; structural connectivity (AIns–NAcc) relates to gambling behavior
  • NAcc activation effects in streaming/music studies: r ≈ 0.460.46 in reported data (Tong et al., 2020) for the relationship between NAcc signal and view frequency

Connections to theory and real-world relevance

  • The MID task and related fMRI findings anchor neuroeconomic theories about how people compute expected value and risk during decision making
  • The endowment effect links affective neural signals (AIns) to valuation shifts when ownership changes
  • Structural connectivity between affective (AIns) and reward (NAcc) regions helps explain individual differences in risk-taking, gambling, and addiction vulnerability
  • Cross-domain studies (music, movies, goods) illustrate the generalizability of reward system signals for predicting real-world choices
  • Critical caution: neuroimaging results should be interpreted within the limits of reverse inference and methodological constraints; meta-analytic tools (e.g., NeuroSynth) enable more robust connections between brain activity and cognitive terms, but do not provide clean one-to-one mappings

Notes on practical and ethical implications

  • Real-time brain signals offer potential applications in marketing, clinical interventions, education, and personalized decision support
  • Ethical considerations include privacy, autonomy, and potential manipulation concerns when decoding or influencing preferences
  • Neuroscience should complement, not replace, behavioural data and theoretical models when informing policy or consumer design