Orbitofrontal cortex, the striatum & dopamine

Week 10

The role of the orbitofrontal cortex (OFC) in descion making

• ‘Hot’ cognition

• Reward-based decision making

• Somatic marker hypothesis

• Complex emotions

general consensus about regions in lateral/medial PFC vs OFC

• lateral/medial PFC are doing something different from orbitofrontal cortex

  • PFC = cold cognition

  • OFC = hot cognition

IOWA gambling task

• p were presented w/4 virtual decks of cards on a computer screen

• told that each time they choose a card they will win money

• every so often a card would lose them money

• goal of the game is to win as much money as possible

• decks differ from each other in the number of trials over which the losses are distributed

• Thus, some decks are "bad decks", and other decks are "good decks", because some will lead to losses over the long run, and others will lead to gains

galvanic skin response

• measure of sweating response

• gives insight to how body is responding to rewards

• is taken as a measure of how anxious a subject is

Role of orbitofrontal cortex in cognitive control (Bechara, 1994) (Behavioural)

• IOWA gambling task to those with lesions

• found normal controls generally learned to avoid the risky decks & choose the decks that would give them an overall profit

• P w/lesions to occipital, temporal & dorsolateral PFC regions showed equivalent performance, generally preferring the advantageous decks.

• P w/OFC lesions showed an overall preference for risky decks, resulting in an overall loss of money

  • continual preference for risky decks

    • they were unable to learn about response-reward contingencies

Role of orbitofrontal cortex in cognitive control → skin conductance response (Bechara et al. 1996)

• tested the skin conductance response (SCR) of OFC lesion patients performing this task

• found that SCRs during reward & punishment were roughly equivalent in patients & controls

• However, anticipatory SCRs (measured between the choice of deck and the receipt of the reward/punishment) were greatly reduced in patients with OFC lesions

  • P failed to activate biasing signals that serve as value markers in the distinction between choices with good and bad future outcomes

• one of the first studies to show role of the orbitofrontal cortex in value-based decision making

Somatic marker hypothesis (Damasio)

• emotional information (physiological arousal) is needed to guide decision-making

• somatic markers are the bodily reactions to emotional stimuli

  • e.g skin conductance response

• OFC supports learning of associations between somatic markers & complex situations

• OFC can then use this information to assess likely outcomes of behavioural choices

• suggesting that emotional signals linked to physiological responses influence decision-making, particularly in uncertain situations

• OFC plays a crucial role in associating these somatic markers with outcomes → therefore if lesioned unable to guide decision making

Problems w/Iowa gambling task data

• cards are presented in a fixed order that induces a preference for risky decks

  • wins experienced early, losses experienced later

    • leads to pattern of behaviour

• in order to perform well tasks, P must at some point overcome this preference to switch to choosing the good decks

  • process of inhibition & switching → reversal learning

reversal learning

• the ability to adapt behavior when previously learned reward contingencies (rules) change, requiring individuals to flexibly adjust their actions based on new information

Changes made to IWOA test to change the way they get losses (Fellows et al, 2005)

• standard version (loss experienced later) vs variant (losses experienced earlier) which doesnt require no reversal learning

• VMF P are impaired relative to controls on standard version of the IGT, in which risky losses are experienced only later (reversal learning required)

• impairment disappears in the variant of the IGT in which risky losses are experienced earlier, and therefore, reversal learning is not required.

• Suggests deficit is in reversal learning rather than value-based decision making more broadly.

Role of OFC in reversal learning → neuropsychological evidence (Fellows et al, 2003)

• investigated precise role of OFC in value-based decision-making

• reversal learning task → P learn which card predicts a reward and then the contingencies are shifted. 

• P w/DLPFC and ventromedial (OFC) lesions. 

• DLPFC were unimpaired.  OFC had specific deficit at the reversal stage

  • They could learn initial response contingencies but had problems learning about new contingency.

• researchers argued that OFC is necessary not so much for learning about stimulus-reward contingencies but for unlearning of established associations

• OFC represents “specialized neural circuitry for rapidly unlearning or suppressing the influence of an established stimulus-reinforcement association”  

Problems w/reversal learning hypothesis of OFC function

• Monkeys w/OFC lesions do not always show reversal learning deficit

• In one study, monkeys had to select either a large or small reward (1 vs 4 peanuts)

• if monkeys selected small reward → given large one

• if monkeys selected large reward → given small one

• monkeys with OFC lesions perform just as well → so reversal learning doesnt fit

Why do monkeys w/OFC lesions not show deficits on this type of reversal learning?

• Task requited overcoming an innate preference for larger food rewards → different front standard reversal learning which involves learning & impression of arbitrary stimulus-outcome relationships

• monkeys took longer to learn this task (50 sessions) to overcome the innate preference for larger whereas reversal learning takes place over much shorter periods

• OFC may only be required for reversal learning when stimulus-outcome computations are made ‘on the fly’ & impression involves arbitrary stimulus-outcome relationships

overall….

• Recent perspectives on OFC function emphasise the computation of value but….

• OFC serves as an ‘accountant’

  • converting information about outcomes (probability, magnitude, costs etc.) into a ‘common neural currency’ on which to base choices

• This may account for the role OFC seems to play in generation of complex emotions such as regret

OFC & counterfactual thinking (regret) - neuropsychological evidence (Camille et al, 2004)

• evidence that this function plays a role in more complex emotions such as regret. 

• P chose 1 of 2 wheels w/different chances of winning

• either got

  • partial feedback

    • found out what they won & they also know what the possibilities were if they chose the other wheel, but they don’t actually know what they would have won if they chose the other wheel.the possibilities

  • full feedback

    • where they found out what they won & also what they would have won if they had chosen the other option → regret

• asked p to rate their level of satisfaction w/outcome

• healthy P showed higher ratings of satisfaction when they win compared to lose

  • show much higher ratings of satisfaction when they find out they would have won less or lost more

  • show much lower ratings when they find out they would have won more/lost less

• OFC patients did not show this pattern of counterfactual thinking

  • didn’t modulate their ratings according to whether alternative wheel would have been a win or lose.

• Suggests a key role for the OFC in being able to experience the consequences of winning and losing, but more importantly to integrate information from different sources of outcome

• OFC important role in mediating how we feel based on the decisions we make

Summary of the role of the OFC in decision making

• Facilitating effective decision making based on knowledge of the value of different actions

• Suppressing actions based on recently learned action-outcome relationships

• OFC seems necessary for value computations only when based on new information that changes rapidly

• Ability to ‘mentally simulate’ information about predicted outcomes seems key here

The striatum (basal ganglia)

• collection of old, subcortical structures including caudate, putamen, globus pallidus & ventral striatum

• loops seem to occur in parallel

• different loops connecting different PFC regions & hypothesised to play different roles

• e.g a reward processing loop that connects ventral striatum to OFC & an executive control loop that connects the DLPFC to dorsal striatum

The role of the striatum in executive function

• Dopamine: the ‘pleasure chemical’?

• Role of dopamine in executive function

Dopamine (DA)

• a neurotransmitter

• produced by dopaminergic neurons in ventral tegmental area (VTA) & substania nigra (SN) in midbrain

• there several different types of DA recports (D1 to D5)

  • D1 receptors much more abundant in PFC

  • D2 receptors are more abundant in striatum

• dompamingeric signalling occurs via several different pathways

dopamine pathways

• mesolimbic pathway

• mesocortical pathway

• nigrostriatal pathways

• tuberoinfundibular pathway

dopamine - the pleasure chemical?

• dopamine does not do that

• not a pleasure chemical but is involved in reward processing and motivation.

Function of dopamine

• role of DA in addiction is well-established

• modern theories of DA emphasise learning & motivation over pleasure (hedonic impact of rewards)

• drugs of abuse enhance DA function by acting on midbrain neurons to increase concentrations of DA

• DA neurons in the striatum encode reward prediction error

  • discrepancy between expected & gained reward (Schultz, 1998)

  • dopamine mathematically code the difference between expected reward and actual reward

    • key for learning

• prediction errors play a key role in development of addiction

Role of DA in working memory & executive functions (Under recognised!)

• Brozoski (1979) – dopamine depletion in PFC of monkeys causes a deficit in spatial working memory nearly as severe as complete ablation (removal)

• Parkinson’s Disease → Gradual loss of dopamine neurons in the nigrostriatal pathway

  • Primarily a motor disorder - bradykinesia (slowed movement), akinesia (impairment of voluntary movement) & tremor

  • Currently no cure although treatments include L-dopa (precursor of dopamine) & more recently deep brain stimulation

• Studies in the 1980s/90s also found deficits in executive control functions in PD

  • Planning, working memory, attentional set-shifting (WCST), Stroop.

How does dopamine influence cognitive control

• One theory suggests that dopamine might play different roles depending on the site of action

  • D1 receptors are more prevalent in PFC → function linked to stability of representations

  • D2 receptors are more prevalent in striatum → function linked to flexible behavior

• Dopamine might achieve a balance of stability & flexibility by exerting different effects on striatal & PFC activation

  • too little flexibility = OCD

  • too much flexibility = ADHD

Effects of methylphenidate (‘Ritalin’) on brain activation during reversal learning (Dodds et al, 08)

• Examined effects of methylphenidate (commonly used to treat ADHD) on brain activation during reversal learning

• Methylphenidate is a dopamine reuptake inhibitor

• gave healthy P a probabilistic reversal learning task in the MRI scanner.

• P were presented w/2 abstract visual patterns

• Using trial-and-error feedback, P must discover which of the 2 patterns is correct (the subject's choice is indicated here with a small, black arrowhead)

• Feedback (a green smiley face or red sad face) is presented as soon as P has chosen one of the patterns w/a left or right button press

• After several correct trials, contingencies switch & P must switch to selecting the other pattern.

• effects of methyphenidate on switch related brain activation

  • drug is dampening the switch control therefore showing that dopmaine is doing something which allows for switching & response to new stimulus

  • non switch errors (don’t receive feedback) more activation in prefrontal cortex

    • maybe helping remain on task

  • results showed a strikingly clear effect → only region that showed such modulation was the putamen (part of the striatum)

  • results show a key role for striatal dopamine in mediating cognitive flexibility.

dopamine reuptake inhibitor

• after dopamine is released into the synapse

• it prevents its re-absorption

• leading to an overall increase in extra-cellular dopamine levels.

Gating role for basal ganglia in cognition - flexible updating of current goal states into PFC (Van Schouwenberg et al., 2010)

• Studies have led to models of the basal ganglia (striatum), in which basal ganglia acts as a ‘gate’ that allows or prevents new goal states from entering PFC.

• current goal states (or tasks or stimulus-response contingencies) are represented by coalitions of neurons in PFC

  • controls what pfc stimuli is paying attention to

• representations remain stable while the subject is focused on the current task or goal

• issues in basial ganglia can result in pathological issues

  • too fast switching = ADHD or too slow = OCD

• But when the reward contingencies change, or new information becomes available, requiring a switch in responding, dopamine signalling in the basal ganglia prevents responding to the previously relevant stimulus and allows the establishment of a new PFC representation, enabling a different stimulus to gain control of responding.

• Evidence consistent with a dual role for dopamine in maintaining stability of responding (PFC) and enabling flexible switching (striatum)

• Perhaps dopaminergic signaling in the striatum can account for individual differences in traits (e.g. impulsivity) that are risk factors for ADHD

Cools et al examined effects of bromocriptine (D2 receptor antagonist) on switching behaviour & neural activation in the striatum

• examined effects of bromocriptine (D2 receptor antagonist) on switching behaviour & neural activation in the striatum

  • more specific than other study

• Colour of fixation cross instructed subjects to encode faces or scenes

• Trials were either switch (attend to different stimulus than the previous trial) or nonswitch (attend to the same stimulus as previous trial)

• did a working memory task in which they focused on faces & scenes

• catergoised to high or low impuslives

• behavioural data

  • switch cost (switching between memory & faces) cost e.g how much worse they get

  • high impulsive seemed to have high switch cost relative to low impulse

  • bromocriptine for reduce switch cost in people who are high impulsive

  • but less so in low impusl

• this was the same in FMRI

• stronger effect in high impulsive

• forms a link between dopamine in striautm which allows for switch changing

• maybe one of the problems in ADHD is the issue with a problem in stiratum or dopmaine receptors

Summary – role of striatum and dopamine in executive function

• Striatum plays a key role in executive function

  • controls access into PFC

• Parkinson’s Disease involves a deficit in cognitive flexibility suggesting a role for striatal dopamine

• Effects of dopamine on executive function may depend on site of action

• Striatum – Cognitive flexibility

• PFC – Cognitive stability