Comprehensive notes on brain circuits of Pavlovian and instrumental conditioning, PIT, and extinction

Pavlovian conditioning: brain circuits and context

Goal of lectures: overview of brain regions mediating Pavlovian and instrumental conditioning and their interactions; focus on animal models (mostly rodents) with translation to humans when possible.
Pavlovian conditioning basics:
- Organisms learn predictive relationships between a conditioned stimulus (CS) and a biologically significant event (unconditioned stimulus, US).
- Two-dimensional framework: form depends on (1) CS-US association type and (2) motivational properties of the US.
- Four forms:
- Appetitive excitatory: CS predicts arrival of a positive event (e.g., food).
- Appetitive inhibitory: CS predicts omission of a positive event.
- Aversive excitatory: CS predicts arrival of a negative event (e.g., shock).
- Aversive inhibitory: CS predicts omission of a negative event.
- Focus in many slides: aversive excitatory Pavlovian conditioning (fear conditioning).
CS processing depends on CS modality:
- Auditory CS (tone/click): processed by auditory thalamus and auditory cortex; rapid thalamic processing, cortex provides detailed information.
- Visual CS (light): processed by visual thalamus and visual cortex.
- Context CS (conditioning chamber): processed by the hippocampus.
- Key PMIDs cited: auditory CS processing (PMID: 3703252, 7891169); visual CS (PMID: 7891169); context CS (PMIDs: 1585183; 1590953; 7472506).
Hippocampus, context processing, and dual-process theory:
- Hippocampus processes information about physical contexts; located in medial temporal lobe; etymology: Hippocampus = sea-horse.
- Dual-process theory of context processing (three key assumptions):
  1) Context can be processed as elemental features in cortex.
  2) Context can be processed configurally by the hippocampus (configural representation).
  3) Default preference for configural representations due to advantages like pattern completion (retrieving all elements from partial cues).
- Experimental evidence (context fear conditioning study with hippocampal lesions): four groups (Gp1–Gp4) with sham vs. hippocampal lesions before/after conditioning; conclusions:
- Conditioning robust in all groups (fear learning occurs).
- Retrieval depends on hippocampus: intact hippocampus during conditioning but lesioned after (Gp4) prevents retrieval; hippocampus crucial for retrieval of configural context; lesions before conditioning (Gp2) still allow fear learning via elemental cortical cues but test freezing may vary.
- Configural representations and the dual-process theory explain why context fear can be preserved or disrupted depending on when the hippocampus is lesioned relative to conditioning and testing.
Dual-process theory: Three assumptions (context processing)
- Contexts can be encoded as elemental features in cortical areas.
- Contexts can be encoded configurally in the hippocampus.
- Configural representations are favored by default due to advantages (pattern completion).
The amygdala and CS-US association:
- Basolateral amygdala (BLA) is a convergence zone for CS and US information; critical for forming CS-US associations.
- Subnuclei roles:
- Lateral amygdala (LA): involved in fear conditioning to discrete cues; receives input from thalamus/cortex.
- Basomedial amygdala (BM) and basal amygdala (BL): connected to hippocampus; involved in context fear conditioning (BL).
- The central amygdala (CeA) serves as an output structure coordinating fear responses; role examined in extinction and habit-related processes.
BLA in fear conditioning: evidence from temporary inactivation and NMDA receptor manipulation
- Temporary inactivation of BLA (e.g., muscimol) disrupts acquisition and retrieval/expression of fear memories.
- If inactivation occurs before conditioning, acquisition is impaired; if before testing, retrieval is impaired; if before conditioning and testing, both phases are impaired.
- NMDA receptors in BLA (NR2B subunit) involvement:
- Ifenprodil (NR2B antagonist) before conditioning impairs acquisition; before testing does not impair retrieval/expression.
- Suggests NMDA receptor activation in BLA is necessary for acquisition but not retrieval/expression.
- Protein synthesis in BLA is required for consolidation of fear memories (anisomycin infusion after CS-US pairing impairs fear at 24 h).
Memory engrams and tagging neuronal ensembles:
- Engram: physical/biochemical changes in brain representing storage of a memory; retrieved by reactivation of the same neuronal ensemble.
- Genetic tagging tools allow tagging of neurons active during learning (learning-event window is controlled and durable tagging enables later manipulation).
- Studies show fear engrams in the BLA with differential engagement of basal vs. lateral amygdala when testing context vs. discrete fear, supporting distributed memory storage within amygdala networks.
Central amygdala (CeA) and serial vs. parallel models of amygdala function:
- Traditional serial model posited CeA as output relay for fear expression; CeA purely drives expression, LA/BLA drive acquisition.
- Newer evidence shows CeA participates in acquisition as well as retrieval/expression; CeA and BLA interactions shape fear memory across acquisition and expression phases.
Summary of Pavlovian fear conditioning circuitry (contextual synthesis):
- Context CS processing: hippocampus; CS modality determines initial sensory processing (auditory thalamus/cortex; visual thalamus/cortex).
- BLA integrates CS-US associations; LA supports cue-specific fear; BL supports context fear via hippocampal input.
- CeA contributes to fear expression and, under revised models, to acquisition of fear memories.
- Extinction, fear restoration, and fear-related memories rely on BLA, PL, IL, ITC, and hippocampal interactions.

Instrumental conditioning: brain circuits

Instrumental conditioning (operant conditioning):
- Learning that actions lead to consequences, guiding voluntary behaviors.
- Two forms of instrumental behavior:
- Goal-directed actions: actions controlled by their outcomes and their contingencies; sensitive to outcome value and causal relationships.
- Habits: actions performed automatically, with little regard to current outcomes; less sensitive to outcome value or causal changes; can be maladaptive.
- Everyday examples: driving, pressing a lift button, etc.
Experimental manipulation of instrumental learning:
- To induce habit formation, researchers use extensive training or manipulate reinforcement schedules.
- Schedules of reinforcement:
- Ratio schedule: outcomes delivered after a number of actions (e.g., every 10 presses); promotes goal-directed actions because action-outcome rate is correlated with outcome delivery.
- Interval schedule: outcomes delivered after a time interval, independent of action count; promotes habitual actions because action rate does not predict outcome delivery.
Neuroanatomical substrates of goal-directed actions:
- Medial prefrontal cortex (mPFC) and dorsal striatum (DS) are key; prelimbic cortex (PL) links to goal-directed control; the posterior dorsomedial striatum (pDMS) is crucial for acquisition and updating actions based on current contingencies.
- Basolateral amygdala (BLA) contributes to encoding outcome value for goal-directed actions.
- PL lesions before training disrupt acquisition of goal-directed actions; PL lesions after training do not disrupt retrieval/expression.
- pDMS lesions disrupt acquisition and retrieval/expression of goal-directed actions; anterior DMS lesions show less consistent effects.
Study results for PL, pDMS, and BLA in goal-directed actions:
- PL role: necessary for acquisition; not required for retrieval/expression.
- pDMS role: necessary for acquisition and retrieval/expression; supports updating actions with current contingencies.
- BLA role: encoding of outcome value necessary for acquisition; retrieval/expression is not solely dependent on BLA during testing; BLA encodes incentive value that informs goal-directed action selection.
Summary of instrumental goals and brain regions (goal-directed):
- PL: acquisition of goal-directed actions.
- pDMS: acquisition and retrieval/expression of goal-directed actions.
- BLA: encoding outcome value; supports acquisition of goal-directed actions but not simply retrieval.
- vmPFC/caudate homology in humans: vmPFC and anterior caudate serve roles analogous to rodent PL and pDMS in goal-directed actions.
Habits and related circuitry:
- Infralimbic cortex (IL) and dorsolateral striatum (DLS) implicated in habitual actions.
- Central amygdala (CeA) also implicated in habit learning and may interact with DLS via circuits involving IL and CeA.
- Two parallel, dissociable systems for goal-directed vs habitual actions:
- PL and pDMS support goal-directed actions.
- IL and DLS support habitual actions.
- CeA involvement in the acquisition of habitual actions demonstrated by anterior CeA lesions restoring sensitivity to outcome devaluation after overtraining.
Interactions and connectivity:
- IL and DLS do not share strong direct anatomical connections; communication likely mediated via other regions (notably CeA).
- Disconnection experiments show CeA-DLS interactions are important for habit formation; contralateral lesions disrupt this interaction and can restore sensitivity to outcome devaluation, suggesting CeA-DLS pathway involvement in habit acquisition.
Distinction of roles and implications:
- PL is necessary for acquisition of goal-directed actions; IL is necessary for acquisition of habitual actions.
- The pDMS is necessary for acquisition and retrieval/expression of goal-directed actions, while PL is not essential for retrieval/expression once learned.
- The DLS is necessary for retrieval/expression of habitual actions.
Human relevance and translation:
- Homologies: vmPFC and anterior caudate nucleus in humans for PL/pDMS; putamen for DLS.
- Dysfunction in these circuits implicated in disorders such as schizophrenia, substance abuse, obesity, and OCD due to imbalanced control between goal-directed and habitual systems.
Summary of key brain regions for instrumental conditioning:
- Goal-directed: PL, pDMS, BLA (outcome value encoding).
- Habits: IL, DLS; CeA interacts with habit circuits; anterior CeA supports acquisition of habitual actions.
- Parallel and competitive systems underlie goal-directed vs habitual control.

Pavlovian-instrumental transfer (PIT)

Definition and forms:
- PIT: Pavlovian stimuli influence the performance and selection of instrumental actions.
- Two forms: general PIT and specific PIT.
General PIT:
- A Pavlovian CS predicting a food outcome increases overall motivational arousal, enhancing instrumental actions that procure food in general.
- Sensitive to motivational state: abolished when the organism is sated (general PIT relies on general motivational properties of the predicted outcome).
- Neural basis: CeA and nucleus accumbens core (NAc core) implicated for general PIT.
Specific PIT:
- A Pavlovian cue predicting a specific outcome biases choice toward the instrumental action that earns that same outcome.
- Survives changes in motivational state and outcome devaluation (sensory-specific properties matter more than general value).
- Neural basis: BLA, pDMS, and nucleus accumbens shell (NAc shell).
The 3-CS PIT design and findings:
- Pavlovian stage uses S1-O1, S2-O2, S3-O3; instrumental stage uses A1-O1 and A2-O2; transfer test measures lever pressing during S1/S2/S3.
- S1 and S2 drive specific PIT (S1 boosts A1; S2 boosts A2); S3 drives general PIT (boosts both A1 and A2).
- Classic finding: S1/S2 produce lever-specific biases, S3 produces generalized facilitation.
Brain circuitry in PIT: amygdala and nucleus accumbens subdivisions
- BLA encodes sensory-specific properties of outcomes; crucial for specific PIT.
- CeA encodes motivational properties; crucial for general PIT.
- NAc shell: critical for specific PIT.
- NAc core: critical for general PIT.
- DLS/pDMS contributions to PIT: DLS involved in performance aspects of PIT; pDMS contributes to outcome-specific information for PIT in certain contexts.
Manipulations and findings:
- BLA lesions: abolish specific PIT while preserving general PIT; CeA lesions do the opposite (abolish general PIT but spare specific PIT).
- Shell vs core inactivation:
- Shell inactivation disrupts specific PIT; core inactivation disrupts general PIT.
- DLS and pDMS in specific PIT:
- DLS inactivation reduces performance but leaves some specificity; pDMS inactivation abolishes specific PIT, consistent with its role in goal-directed information processing.
Human relevance and clinical implications:
- PIT mechanisms translated to humans with similar brain circuitry; fMRI supports analogous networks in humans.
- General vs specific PIT implicated in eating and addiction; individual differences and disorders (e.g., obesity, substance use disorders) linked to PIT processes.
Optional design variants and deeper points:
- Three-condition PIT (3-CS PIT) enables dissociation of general vs specific PIT within the same subject.
- Motivational manipulations (hungry vs. sated) reveal distinct dissociations:
- Hungry state: both general and specific PIT evident;
- Sated state: general PIT abolished; specific PIT persists, indicating differential reliance on motivational vs sensory properties.
Summary of PIT: two parallel, dissociable mechanisms with distinct neural substrates (CeA/NAc core for general PIT; BLA/pDMS/NAc shell for specific PIT), and dual contributions to behavior (performance vs choice).

Extinction of Pavlovian fear conditioning

Concept of extinction:
- Extinction is new learning: CS is presented without the US, reducing conditioned responses (CRs) through the formation of CS-noUS inhibition; extinction memory competes with original CS-US memory.
- Extinction is not erasure; fear memories can be restored by context, time, or stress (renewal, spontaneous recovery, reinstatement).
Fear restoration phenomena:
- Renewal: extinguished fear reappears in a different context (AAB, ABA, ABC forms).
- Spontaneous recovery: fear returns after a delay post-extinction.
- Reinstatement: fear returns after exposure to the US or a stressor post-extinction.
Brain circuits in extinction (fear extinction):
- BLA crucial for acquisition of extinction; lesion or inactivation during extinction impairs extinction learning.
- Medial prefrontal cortex (mPFC) subdivisions:
- PL: necessary for expression of conditioned fear; not strictly required for acquisition of extinction.
- IL: crucial for consolidation and retrieval/expression of extinction; inactivation of IL disrupts extinction retrieval; optogenetic silencing of IL at test increases freezing, indicating IL involvement in extinction retrieval.
- IL interactions with intercalated cells (ITC) in the amygdala and BLA to suppress fear via inhibitory pathways; ITC lesions impair extinction retrieval.
Extinction memory consolidation and retrieval/expression:
- BLA and IL work together to form and retrieve extinction memories; IL may activate ITC to suppress fear responses.
- Post-extinction manipulations (e.g., BLA inactivation after extinction) can impair consolidation of extinction memory (fear persists at test).
- Optogenetics: IL silencing at test impairs retrieval/expression of extinction, demonstrating the necessity of IL activity for fear extinction expression.
Role of BLA neuron subpopulations in fear extinction:
- Fear neurons: high activity after fear conditioning (sharp CRs).
- Extinction neurons: elevated activity during extinction; silencing extinction neurons during extinction increases fear, suggesting extinction neuron activity is important for extinction learning.
The hypothesized circuit for extinction:
- IL drives ITC interneuron activation, which in turn suppresses amygdala output to reduce fear responses.
- BLA is necessary for both acquisition and consolidation of extinction; mPFC regions integrate with amygdala networks to regulate extinction learning and expression.
Clinical relevance:
- Cue exposure therapy for anxiety disorders relies on fear extinction principles; understanding extinction circuitry informs treatment and relapse risk.

Connections across sections and overarching themes

Two independent and parallel memory systems for Pavlovian and instrumental learning exist, which compete for behavioral control.
- Goal-directed vs habitual systems operate in parallel; disruption in one system can shift control to the other.
Extinction interacts with PIT and instrumental learning through context, memory retrieval, and motivational state.
The amygdala, prefrontal cortex, striatal circuits, and hippocampus form a broad network underpinning learning, memory consolidation, retrieval, and behavioral control across Pavlovian, instrumental, and extinction processes.
Translational relevance:
- Human studies (fMRI, vmPFC/anterior caudate for goal-directed actions; putamen for habitual actions) align with rodent data.
- Dysfunction in these circuits may contribute to schizophrenia, substance use disorders, obesity, OCD, anxiety disorders, and other conditions.

Quick reference: key brain regions by function

Pavlovian fear conditioning:
- CS processing: Hippocampus (context), Auditory/Visual thalamus and Cortex (CS modality).
- CS-US convergence: Basolateral amygdala (BLA).
- Fear expression: CeA and downstream targets; extinction involves IL, PL interactions with BLA/ITC.
Instrumental goal-directed actions:
- PL: acquisition of goal-directed actions.
- pDMS: acquisition and retrieval/expression of goal-directed actions.
- BLA: encoding outcome value.
Habits and automatic actions:
- IL: acquisition of habitual actions.
- DLS: retrieval/expression of habitual actions.
- CeA interacts with DLS in habit acquisition; anterior CeA critical for habit acquisition.
Pavlovian-instrumental transfer (PIT):
- Specific PIT: BLA + pDMS + NAc shell; behavioral bias toward action that earned the same outcome as the cue.
- General PIT: CeA + NAc core; generalized facilitation of actions procuring food.
- DLS: contributes to performance aspects of PIT; pDMS contributes to outcome-specific information in PIT.
Extinction:
- Extinction is new learning (CS-noUS); extinction memory consolidation involves BLA and IL; IL may activate ITC to suppress fear.

Notable study notes and identifiers (selected PMIDs)

Context processing and hippocampus in context fear conditioning: PMID: 9404635
BLA involvement and pharmacology in fear conditioning: PMID: 9267646; PMID: 11517276; PMID: 10974093
NHDA receptor NR2B in BLA and fear acquisition: PMID: 11517276
Engrams and tagging neuronal ensembles: PMID: 38664582; PMID: 31896692; PMID: 26335640
Extinction circuitry and IL/PL roles: PMID: 20962768; PMID: 18772253; PMID: 24908482; PMID: 26354044
ITC and extinction retrieval: PMID: 24908482
PIT circuitry in amygdala and NAc subdivisions: PMID: 20385164; PMID: 15673677; PMID: 26970240
DLS and pDMS roles in specific PIT: PMID: 15078120
Nucleus accumbens core vs shell in PIT: PMID: 26970240
Human homologues of rodent PIT circuits: vmPFC/anterior caudate vs putamen

Glossary of symbols and concepts used in these notes

CS: conditioned stimulus; US: unconditioned stimulus; CR: conditioned response.
V_{CS-US}: strength of the CS-US association.
CS-US memory: the memory formed linking CS to US.
CS-noUS memory: extinction memory formed when CS occurs without US.
Extinction memory retrieval/expression: the use of extinction memory to suppress the original fear response during test.
Outcome devaluation: process by which the value of a reward is reduced to test if behavior is goal-directed or habitual.
Contingency degradation: procedure reducing the causal link between action and outcome to test goal-directed control.
Specific PIT: Pavlovian cue enhances action that earns the same specific outcome.
General PIT: Pavlovian cue enhances action regardless of the specific outcome.
ITC: intercalated cells of the amygdala; implicated in extinction retrieval via inhibitory pathways.
Engram: the assumed neural substrate of a memory trace, identifiable via tagging of active neurons during learning.
Optogenetics: technique enabling precise temporal control of neural activity using light-sensitive proteins.
Science-based therapeutics: cue exposure therapy parallels fear extinction processes in clinical contexts.