PS263 lecture 12: Reward

Reward

type of information, usually carried by sensory stimuli, that can cause approach and appetitive behaviours, can reinforce movements (cause them to be repeated), and can induce positive emotions 

Punisher (negative reinforcer)

information usually carried by sensory stimuli that can cause and train avoidance behaviours and can induce negative emotions 

Liking (emotion)

How good something feels

• autonomic nervous sys/visceral nervous sys - body reacts when something feels pleasant 

• changes in facial expression - show liking reactions (licking lips for something sweet)

• subjective reports of positive feelings - saying “i like this, this feels good”

Wanting (motivation)

The drive or desire to obtain something. dopamine related 

• “Stimulus–response” behaviors (approach)
  Automatic actions triggered by cues.
  Example: seeing fries → reach for them.

• “Instrumental” behaviors to obtain reward
  Goal-directed actions.
  Example: working harder to earn money, pressing a lever to get food.

• Outcome-based plans
  More deliberate strategies.
  Example: planning to go to a specific bakery because you crave their pastries.

Reinforcement (learning)

The process by which rewards teach you to do something again

• Learning to value stimuli/places/situations
  Example: You associate a café with comfort → you return.
  (Classical conditioning.)

• Learning to perform certain movements
  Example: You learn which actions produce a reward.
  (Operant conditioning.)

• Learning outcomes
  You form expectations: “If I do X, Y will happen.”
  This strengthens future behaviour.

Classical Conditioning (pavlovian)

if you repeatedly present a valued stimulus AFTER a neutral stimulus, an animal will show autonomic and motor responses to the neutral stimulus 

• US: Food

• UR: Salivation to food

• NS: Bell

• Pairing: Bell + food → salivation

• CS: Bell

• CR: Salivation to bell 

Operant conditioning

animals learn to repeat actions that lead to a valued stimulus by associating behaviors with consequences, either rewards or punishments.

• at first = actions are specific to the valued stimulus 

• gradually = become independent of the stimulus 

action-outcome associations

actions guided by expectations, A goal-directed link between a specific action and its expected consequence.

• You choose an action because you want a particular outcome.

• Requires prediction, evaluation, and the idea:
  “If I do X, I will get Y.”

stimulus-response associations

habit based automatic link between stimulus and a behaviour

• Formed through repeated reinforcement.

• Does not require thinking about the outcome anymore.

• Driven by cues in the environment.

wanting vs liking: self stimulation 

where a lady self-stimulated throughout the day with an electronic zapper. at first she liked it but over time, she started disliking it but the “wanting” remained, causing her continue to zap herself even when unpleasant.

which regions of the brain process reward information?

• frontal cortex (medial and orbitofrontal)

• basal ganglia (ventral striatum/nucleus accumbens)

• amygdala 

• neuromodulators 

  • dopamine 

  • all others

orbitofrontal cortex 

stimulus values - how good or bad a specific stimulus is (particularly positive)

• secondary taste cortex 

  • this is where taste is evaluated, with information brought up by the primary taste cortex and determines its hedonic value (how pleasant it is)

    • Chocolate → OFC encodes how good it tastes.
      If you’re full, the OFC lowers its value

Medial frontal (anterior cingulate) cortex 

positive and negative values of situations and actions 

• region involved with action evaluation, not just stimulus pleasure 

• includes

  • cost vs. reward

  • effort required

  • risk

  • consequences

  • conflict between choices

• ex: You want chocolate, but you know you should study.
  The ACC evaluates the trade-offs and motivational value of each action.

Basal Ganglia

A network of interconnected subcortical brain regions involved in movement, motivation, reward learning, habits, and action selection.

• receives major input from cerebral cortex 

Where do the basal ganglia send outputs to?

to the brainstem and back to the cortex through the thalamus.

What is the cortical–basal ganglia–thalamic loop?

A closed circuit: Cortex → Basal Ganglia → Thalamus → Cortex.

Striatum

Entry point for information into the basal ganglia. All major cortical input flows into the striatum before being processed and sent back through the loop.

• Cortex → Striatum → Basal Ganglia → Thalamus → Cortex

Ventral striatum/Nucleus accumbens

part thats crucial for reward processing

• receives information about stimulus value (how good something is)

• integrates emotional and motivation signals

• involved in wanting, motivation and reinforcement learning 

in short, tells you “this is rewarding, go get it”

Dorsal Striatum

region involved in movement, habits and action selection

• helps you choose actions 

• helps form habits (stimulus → response)

• important for both motor control and decision processes 

Dopamine

is a neurotransmitter involved in

• reward

• motivation (“wanting”)

• reinforcement learning

• movement

• habit formation

acts on metabotropic receptors such as D1 and D2

D1 receptors

excitatory

D2 receptors

inhibitory 

Where does dopamine come from in the brain?

1. substantia nigra

2. ventral tegmental area (VTA)

Substantia Nigra

• sends dopamine to dorsal striatum

• important for movement and habit formation

• degeneration leads to Parkinson's disease.

  • substantia nigra → dorsal striatum → movement

Ventral Tegmental Area (VTA)

• Sends dopamine to ventral striatum (nucleus accumbens) and to the PFC

• Important for reward, motivation, addiction

  • VTA → Ventral striatum → reward/motivation 

Phasic 

short, rapid bursts of dopamine in response to events

Tonic

persistent, steady levels of dopamine over time

• Constant, low-level firing of dopamine neurons

• Sets the baseline motivation level

dopamine neurons respond most to ______

unexpected reward and unexpected stimuli that predict reward

what causes neurons to decrease firing

unexpected omission of reward

what happens when you learned to predict the reward?

no longer the same dopamine response 

temporal discounting

the tendency to devalue something larger in the future vs something smaller presented in the present.

• PFC activity needed to overcoming impulsive responses

• serotonin system may also be involved in waiting for reward 

prepotent responses 

are automatic responses that require minimal cognitive control and can interfere with rational decision-making.