lesson 5 part one: The Neuroeconomics of Choice: Value and Reward

This lecture is crucial for understanding the concept of making choices.
Last week's recap: Discussed brain anatomy, including the cortex (top layer), limbic system (below the cortex), and basal ganglia (deeper in the brain).
Brain areas involved in reward and value processing: The brain, specifically the basal ganglia, is key. Important components of the basal ganglia include the striatum, which is composed of the caudate nucleus, putamen, and especially the nucleus accumbens.
The nucleus accumbens is involved in processing the perceived value of things in our environment.

Core principle: We choose by maximizing the subjective value or utility of things. In principle, we assign value to everything in our environment.
To choose between options (e.g., choice X and choice Y), one must know the value assigned to each option.
We select the option that offers the highest value or utility.
Mathematical representation: If the utility of X is greater than the utility of Y (U(x) > U(y)), then X is chosen.
This principle holds well for most simple choices, though its universality in complex situations can be debated.
The process involves comparing the value of things and choosing the one with the most subjective value.

Economic perspective: Values are often referred to as 'incentives,' typically expressed in monetary terms, and called 'economic value' or 'utility.'
Psychological and Neuroscience perspective: Values are often termed 'rewards' and relate to concepts like food, sex, social approval, status, image, and identity. These are considered 'psychological' or 'biological needs' (referencing Maslow's hierarchy of needs for context).
Hierarchical nature: Some things are inherently more valuable (e.g., food, security) than others, especially for survival.
Key commonality: All these 'valuable' things, directly or indirectly, promote or increase our chances of survival or 'biological fitness' (the ability to pass down genes to the next generation).

Rewards: Proximate 'approach goals' – things we want to get because they enhance our chances of survival and successful reproduction.
- Examples for humans: Food, social approval, status, money, good image, sex.
- Examples for cats: Specific foods they like, comfortable spaces.
Punishments: 'Avoidance goals' – things we want to avoid because they reduce our chances of survival.
Universal mechanism: This basic mechanism is shared across species (humans, cats, other animals). All organisms seek things that increase their survival chances and avoid things that decrease them.
Fundamental goal of life: To perpetuate life by acquiring rewards and avoiding punishments.

The brain's prime motivation is to predict the value of features in the environment and use this information to compute optimal behavior (approach rewards, avoid punishments).
Cross-species similarity: The neural systems involved in calculating value and reward are remarkably similar across species.
Dopamine: This neurotransmitter is the key linker, universally used in the reward systems of mammals (humans, cats, rats).
Anatomy of the Dopamine System:
- Dopamine molecules are produced in the brainstem, specifically the Ventral Tegmental Area (VTA).
- From the VTA, dopamine information is sent throughout the brain to specific areas, including:
  - The striatum (composed of caudate, putamen, and especially the nucleus accumbens).
  - The prefrontal cortex, particularly the medial prefrontal cortex.
- Schematic images illustrate this system's presence and similar structure in human, monkey, and rat brains.

Neurons: The brain is comprised of cells called neurons, which have a specific shape:
- Cell body
- Dendrites: Little arms used to receive information.
- Axon: A very long wire sticking out, forming the basis of nerves.
Electrical Impulse: When a neuron becomes electrically active (polarized), it sends an electrical impulse (action potential) along its axon towards the cell bodies/dendrites of other neurons it communicates with.
Synapse: Neurons are not directly connected. There's a tiny gap called the synapse between the end of an axon and the receiving neuron.
Chemical Transmission: When an action potential reaches the end of the axon, it releases specific chemicals called neurotransmitters into the synapse.
- In the reward system, the primary neurotransmitter is dopamine.
Modulation: Neurotransmitters act like a 'key in a lock' at receptors on the receiving neuron, opening channels and allowing that neuron to become polarized. This process is not automatic; neurotransmitters modulate the signal:
- They can strengthen, reduce, or even block the signal. The amount of neurotransmitter released determines the strength of the signal propagation.
Different brain systems use different neurotransmitters. For the system dealing with value and rewards, dopamine is central.
The VTA, nucleus accumbens, and ventromedial prefrontal cortex are key areas involved in value processing, all operating under the influence of dopamine.

The brain has evolved from simpler structures to complex ones over time.
Older brain areas are not discarded but are integrated into newer, more complex systems.
Value processing is a core principle for organisms. This functionality has extended and evolved forward into newer brain regions.
Hierarchy of complexity:
- VTA: Relatively simple version of reward processing.
- Ventral Striatum: More complex version.
- Ventromedial Prefrontal Cortex: Most sophisticated version.
Location: This system is predominantly located in the 'ventral' part of the brain (towards the stomach).
Dorsal Dopamine System: While the ventral system handles value, a separate dorsal dopamine system (towards the top of the brain in humans) is involved in motor control. Dysfunction here is seen in Parkinson's disease, where dopamine agonists can aid movement.

The VTA is the primary site where dopamine molecules are produced.
These dopamine-producing neurons project widely to other areas of the value system, including the ventral striatum and ventromedial prefrontal cortex.

A very important and central part of the value processing system.
Components:
- Caudate Nucleus (pinkish-purple)
- Putamen (orange)
- Nucleus Accumbens (green, the very ventral part of the striatum) – this lies at the center of the value processing system.
Historically, the striatum was known for its role in motor control. Its involvement in reward and value processing was discovered relatively recently (around 75 years ago).

Experiment Setup: A rat in a cage has an electrode implanted in its reward system (e.g., ventral striatum). A wire connects the electrode to a lever, allowing the rat to self-stimulate its brain with an electrical impulse.
Observed Behavior: Rats start pressing the button and do not stop. They ignore essential needs like eating or social interaction, pressing the lever ceaselessly until collapsing from fatigue.
Interpretation: The artificial stimulation causes the brain to perceive the action (pressing the button) as highly rewarding or valuable. The system signals