Decision Making in the Brain Final Study Guide

What is decision-making?

Decision making is the cognitive process of selecting a course of action from multiple alternatives. It involves evaluating information, considering potential outcomes, and choosing the best response based on goals, values, and available knowledge.

• Decision making is a thought process that involves a complex interplay of perception and predictions.

  In decision making the brain is interrogating our surroundings and finding answers or making predictions ?

• Decision making is the cognitive process of assessing information to select a course of action among several alternatives.

Examples of decision making in our everyday lives:

1. Whether we are deciding which direction to move our eyes

   or which career path to pursue, decision-making makes up a

   large part of our everyday lives.

   • Although we may be unaware of it, our brains are constantly

   weighing possible options and predicting potential

   outcomes as we navigate through the complex world

   around us.

2. Choosing what to eat – Deciding between cooking at home or ordering takeout.

3. Social decisions – Deciding whether to attend a party or stay home.

4. Financial decisions – Choosing to save or spend money on a purchase.

5. Health decisions – Deciding to exercise or rest when feeling tired.

6. Time management – Choosing whether to study or relax

Why is it important to understand the neural mechanisms of decision making

• Understanding the brain systems and circuits that motivate decisions in nonhuman animals helps us better understand how our own brains make good decisions - or bad ones - and how to make better decisions in the future.

  -developing brains

  -education

• Helps explain how the brain processes information and weighs options.

• Provides insight into mental health conditions (e.g., depression, anxiety) that affect decision making.

• Improves understanding of addiction and impulsive behavior.

• Can inform the development of AI and neural network models based on human decision strategies.

• Aids in creating more effective interventions for cognitive impairments.

Different model systems used to study decision making and their benefits:

1. Humans – Provide direct insights into complex cognitive processes but are difficult to control experimentally.

2. Monkeys

3. Mice

4. Bee

5. Drosophila

Benefits:

• Comparing across species helps identify common neural mechanisms.

• Easier to study genetic, molecular, and neural factors in simple systems.

• Testing models in artificial systems allows for simulated hypothesis testing.

Different types of information that contribute to decisions:

1. Brain integrates evidence from diverse sources (for example, different senses and memory),

• The last time I visited the pond, a rustling sound +angle of the sun + temperature = predator

• A coffee shop visit might factor in time of day, previous knowledge of barista available, time duration taken to reach the coffee shop, a specific scene at the coffee shop that triggers a good/bad memory

2. Assign more or less weight to cues that differ in their reliability,

3. Calculate expected costs and benefits associated with anticipated outcomes,

4. Process elapsed time to meet a deadline or to assess temporal cost, and implement rules (such as deciding on what to decide upon) and policies (balancing accuracy against speed).

Different senses or modalities that allow us to sample the environment:

Vision

Touch

Hearing

Examples of spatial and temporal features in the sensory environment that contribute to decision making:

• Spatial features:

  • What is the color of the traffic light?

  • Location of a sound helping to decide which direction to turn.

  • Distance of an approaching object affecting when to react.

• Temporal features:

  • How long was the light on for ?

  • Timing of a traffic light changing from yellow to red.

  • Rhythm of speech helping to interpret language.

  • Speed of an approaching object influencing the urgency of response.

How is interval timing important in decision making?

• Predicting events – Anticipating when a traffic light will change.

  • Deliberation and planning in the context of decision making as a cognitive process that

    integrates information across time.

    • We progress to consider how temporal expectations of the future modulate perception.

    • (Making predictions)

What is sensation, perception and what is the role of sensory receptors? What is the location of sensory receptors?

Sensation : The process by which the nervous system detects physical events around us.

Perception: The experience we have as we process and interpret information from sensory cells.

Percept is our interpretation of the sensory stimulus

Sensory receptors “receive” the stimulus or respond to the appropriate stimulus.

Local aggregation of sensory receptors that respond to one specific type of stimulus

is called a sensory receptor organ.

Concept of adequate stimuli, explain with an example

Adequate stimulus is light for receptors in the eye. It is the type of stimulus for which a given sensory receptor organ is particularly adapted.

Visual system- retina, pupil, fovea, blind spot, optic tract, primary visual cortex, photoreceptors

Retina

Fovea: center of our gaze, vision is sharpest

Lens: brings things into focus

Photoreceptors

(Light detecting neurons)

• Rods- very sensitive, dim light

• Cones-bright light, colors

Rods and cones work together

to allow us to identify the visual

field.

Visual field is that part of space

that we can see at any one time

without moving our eyes.

What is cross over of information? Why do you think this is important ?

Cross over of information from the left visual field to the right hemisphere in the brain

Crossings reduce the long length of axons

Crossing over keeps x,y,z positions

in the same orientation

Concepts such as depth perception and the importance of binocular cues

Binocular cues

There is some distance between the two eyes (6cm) and that

provides slightly different information from the visual field

(different perspectives) - depth perception

Different ways in brain interrogates the sensory environment to get information such as visual distortions - Linear perspective, motion parallax, size shape color constancy, gestalt psychology

The brain utilizes various perceptual cues to interpret sensory information effectively, particularly in visual processing. These include: 1. Linear Perspective: This refers to the way parallel lines appear to converge as they recede into the distance, creating a sense of depth. 2. Motion Parallax: This phenomenon occurs when closer objects appear to move faster than those further away during motion, providing a depth cue. 3. Size, Shape, and Color Constancy: These are perceptual principles that allow objects to be perceived as constant in size, shape, and color, regardless of changes in viewing conditions or distances. 4. Gestalt Psychology: This theory emphasizes that the human brain organizes sensory information into meaningful wholes, rather than simply responding to individual components. These mechanisms work together, allowing for a more comprehensive understanding of our visual surroundings, helping to mitigate distortions and ensure reliable perception in varying environments.

What is a sound ? Examples

Sounds: Waves of air pressure that hit our ear.

Sample our environment

Who is approaching the door?

How far is the car?

Low frequency vs high frequency sounds

Low frequency = low pitch

High frequency = high pitch

What is the role of the cochlear membrane ? Where are high and low frequency sounds represented on the membrane?

The inner ear consists of the cochlea and this plays an important role in transmitting the sound.

The cochlea consists of fluid. As the fluid vibrates, the cochlear membrane vibrates.

The vibration allows detection of sounds of different frequencies

High frequency = base of the cochlea

Low frequency = apex of cochlea

How do binaural cues help in sounds localization ?

Biaural cues allows us to determine the location of sound. If sound is coming to the left ear first

1. Intensity difference : it will be increased in magnitude in the left compared to the right ear.

2. Latency difference: it reach left ear first compared to the right ear.

What the 4 different touch receptors ?

The four different types of touch receptors are: 1) Meissner's corpuscles - sensitive to light touch and vibrations, found primarily in hairless skin; 2) Pacinian corpuscles - detect deep pressure and high-frequency vibration, located deep in the dermis; 3) Merkel cells - responsible for detecting light touch and texture, found in the epidermis; and 4) Ruffini endings - sensitive to skin stretch and sustained pressure, located in the dermis.

Concept of a receptive field

Receptive field: The region of space where stimuli affect the activity of a cell in the sensory system

Pain Nociceptors Phantom limbs

Pain illustrates the further difference between sensation and perception

Nociceptors are specialized sensory neurons that detect and transmit pain signals to the brain. They play a crucial role in the body's pain perception and response to potentially harmful stimuli. 

Individuals threshold for pain varies, often as a result of the potential outcome.

Phantom Limb: Perception of pain without a stimulus. A person who has parts amputated perceives pain in the missing part.

Concept of labelled line strategy with an example

Labelled line strategy: Specific nerves relay/communicate specific types of sensory information

All neurons send only action potentials to the brain

• The interpretation of the action potentials is important in distinguishing between stimuli

• Labelled line strategy allows the brain to interpret the signals coming from a specific modality.

• Information coming in from a specific modality is treated differently.

• So if you press your eye you see a black spot, that is actually not there. However since the information

is coming though the line that normally carries visual

information it interprets it as visual information.

The brain’s interpretations of the sensory events or

perception is crucial and an important

prerequisite in the process of making decisions.

What is detection threshold. Explain with an example ?

Stimulus intensity varies – weak, strong.

Thresholds to detect them change with experience and age.

The threshold for a sensory cell is a weakest

possible stimulus that still affects that cell’s firing

How much sensitivity do we need ? Will we benefit from being aware of all intensities ?

Neurons can only fire upto a certain amount, thus limiting sensitivity.

Neurons need to operate within a dynamic range

What is psychophysics? How does it enable us to study decision making ?

Psychophysics is the study of the relationship between physical stimulus and the resulting experience and behavior.

These studies typically involve varying the stimulus (intensity) and examining changes in behavior.

What is absolute threshold ? Explain with an example

Absolute Threshold- Lowest intensity that a person can detect half the time. ( not reliable) It serves as a benchmark for measuring sensory sensitivity. For example, in the case of hearing, the absolute threshold might be the faintest sound level that an individual can detect, such as a ticking clock from a distance. This threshold varies among individuals and can change based on factors like age, attention, and environmental conditions.

Why do we have sensory threshold? What is the purpose

Sensory thresholds are crucial for the functioning of sensory systems as they define the limits of human perception. The absolute threshold indicates the minimum level of stimulus intensity required for detection, which helps prevent sensory overload in our daily lives, enabling us to focus on important stimuli in our environments. The difference threshold, or just noticeable difference (JND), allows us to detect changes in stimulus intensity, essential for navigating our surroundings and making informed decisions.

Concepts of Just noticeable difference, Weber’s fraction, sensory adaptation.

Just noticeable difference (JND)

How much should the difference between 2 stimuli be in order to distinguish between them.

• Loudness of a sound

• Intensity of light

• JND can impact how we discriminate between sensory stimuli, thus impacting our decision making ability

• Ernst Weber identified a link between JND and stimulus

Weber’s fraction is the smallest change in the magnitude of a stimulus that can be detected, expressed as a ratio of original stimulus

Weber’s fraction

Weight = 2%

Loudness=10%

Light intensity =1%

Do sensory systems need to be constantly responding to a stimulus? Or is it more efficient to respond to a change in stimulus?

When a stimulus does not change over time, receptors and eventually neurons in the brain stop responding to it or they adapt

Does sensory adaptation cause information to be distorted ?

Its better for receptors to be selective and report only changes in stimuli.

Although if the receptors are adapting then does the visual scene soon fade or become non –existent?

No. Because the eyes make small movements call saccades which brings about a change in the visual field.

Why is sensory adaptation important ?

Sensory adaptation is important because it allows organisms to become less sensitive to constant, unchanging stimuli in their environment.

Signal Detection theory: What is it ? Why is it used ? Use examples to explain. What do the 4 behavioral Outcomes such as Hits, correct rejection, False alarms and misses mean.

Distinguishing between signal and noise is a continual process that is part of the interrogation performed by the brain.

Are there footsteps outside ?

Was it a black or white car that passed by ?

Was that an opossum in my backyard ?

Did my phone vibrate or was it the humming from my computer ?

Our brains are actively interrogating the sensory environment and determining the saliency of an input or whether it is a valid signal.

False Alarms vs Miss

Example: A doctor has to constantly adjust his/her criterion to prevent misdiagnosis. Avoid unnecessary surgery as a result of false alarm and fatality caused due to a missed tumor.

What is interval learning ? Why is it important in the context of decision making ?

Interval learning is a type of learning that occurs when individuals receive feedback or reinforcement at specific intervals of time rather than after each action or decision. This approach helps individuals learn to predict outcomes based on the timing and frequency of their responses. In the context of decision making, interval learning is crucial because it allows individuals to analyze patterns over time and make informed choices based on past experiences. For instance, in behavioral psychology, interval schedules of reinforcement have been shown to be more effective than fixed schedules for maintaining behavior, encouraging individuals to weigh the potential rewards and risks linked to their decisions.

What is learning ? Why is it important for decision making ?

Learning : Acquisition of knowledge, skill, attitudes, or understanding as a result of experience.

Learning impacts future decisions we make and ultimately our behavior.

Learning can result in long lasting behavior

What is conditioned learning ? Use examples

Conditioned learning. It is a form of conditioning where we learn to make associations. These associations are stored in our brains and this allows us to make predictions about the future.

Examples: associating an alarm clock with waking up early.

Going to the grocery store on certain days increases availability of your favorite products

I have learned to associate that if my son does not eat a good dinner then he will wake up in the night.

Evolutionary significance: This form of learning is conserved across species.

(fruit flies, sea slugs, microscopic worms, dogs , horses, primates, humans)

Associative architecture of the brain is one of the most robust and well conserved features of brain evolution—

maybe to aid with learning.

What is non associative learning? Explain the concepts of habituation, dishabituation and sensitization with examples. Why are these phenomena conserved across species ?

1. Non-associative learning is a type of learning that does not involve forming associations between stimuli but rather focuses on the changes in behavior resulting from repeated exposure to a single stimulus. It is primarily divided into three concepts: habituation, dishabituation, and sensitization.

2. Habituation occurs when an organism gradually reduces its response to a repeated, benign stimulus over time. For example, if a person moves to a house near a train station, they may initially be disturbed by the noise of trains but eventually become accustomed to it, leading to a decreased response to the sound.

3. Dishabituation is the restoration of a response to a stimulus after it has been habituated, typically triggered by the introduction of a new stimulus. For example, the train noise may fade into the background, but if a new and louder noise, such as construction work, emerges, the person may become alert to the train noise again, indicating dishabituation.

4. Sensitization is the increased response to a stimulus following a strong or noxious event, even if the stimulus itself is not harmful. An example could be a person who becomes overly sensitive to a quiet noise after having experienced a loud explosion. The previous trauma heightens their awareness and response to sounds that were previously ignored.

Associative learning such as classical conditioning. Concepts of CS, US, CR, UR. What is the role of temporal order in classical conditioning

Classical conditioning, developed by Ivan Pavlov, is a type of associative learning where a neutral stimulus becomes associated with an unconditioned stimulus to elicit a conditioned response.

1. Unconditioned Stimulus (US): This is a stimulus that naturally and automatically triggers a response without prior learning. For instance, food is an unconditioned stimulus because it naturally causes salivation in dogs.

2. Unconditioned Response (UR): This response occurs automatically in reaction to the unconditioned stimulus. In Pavlov's experiment, the dog's salivation when presented with food is the unconditioned response.

3. Conditioned Stimulus (CS): This is a previously neutral stimulus that, after being repeatedly paired with the unconditioned stimulus, gains the ability to elicit a response. In Pavlov's experiment, the sound of a bell, which initially does not cause salivation, becomes a conditioned stimulus after being associated with the food.

4. Conditioned Response (CR): This is the learned response to the conditioned stimulus. Following the conditioning process, the dog will salivate upon hearing the bell alone, which is now termed the conditioned response.

The temporal order in classical conditioning is crucial for the learning to occur. It generally involves the presentation of the CS before the US, followed by a short interval, promoting an association between the two stimuli. The timing and order in which these stimuli are presented affect how quickly and effectively the association is formed. For example, if the bell (CS) is rung a second or two before presenting the food (US), the dog learns to associate the sound of the bell with the imminent arrival of food, leading to the conditioned response of salivation. Conversely, if the US is presented before the CS, the association is less likely to form effectively, demonstrating that the temporal relationship is vital for successful conditioning.

Experiments by Robert Rescorla in understanding the how classical conditioning helped predict the future.

Rats were presented with a tone followed by a shock. Soon rats learned to freeze in response to the tone.

Something salient about shock following the tone. If several tone+shock presentations were done and

in addition several shock only representations were done, then no association was made.

The predictive nature or value of the tone was lost.

The neutral stimulus has to help them predict the future.

Classical conditioning is not restricted to one stimulus and can extend to multiple neutral stimuli. Explain with examples

Second order conditioning- previously learned CS, which elicits a CR repeatedly follows another neutral

Stimulus. Now the second neutral stimulus also elicits a CR.

What is stimulus generalization?

Stimulus generalization is a psychological phenomenon in which organisms display a conditioned response not only to the specific conditioned stimulus (CS) used during training but also to other stimuli that are similar to the CS. This process reflects the brain's ability to recognize similarities between different stimuli, allowing for more flexible and adaptive responses in varying situations.

How does conditioning behavior play an important role in taste aversion?

Examples of taste aversion in the normal world. How this influences our decisions and behaviors.

Blue jay will eat a monarch butterfly but only once. The resulting nausea will prevent it from consuming

it in the future.

Poisonous marine larvae will elicit taste aversion in predators.

Vampire bats that feed on blood and do not encounter toxic food do not develop taste aversion.

What is fear conditioning ? How does that explain the idea of classical conditioning ?

Fear conditioning is a form of classical conditioning where a neutral stimulus (e.g., a loud noise) is paired with an aversive unconditioned stimulus (e.g., a shock), leading to a conditioned fear response to the neutral stimulus. It illustrates classical conditioning by showing how an organism learns to associate a previously neutral stimulus with a significant, often negative, event, resulting in a learned fear response.

Experimental methods used to study decision making fMRI, electrophysiology, gaze following, EEG, pharmacology and optogenetics

he experimental methods for studying decision making include fMRI, which measures brain activity; electrophysiology, which records electrical activity in neurons; gaze following, which examines how people track eye movement to infer intentions; EEG, which records electrical activity from the scalp; pharmacology, which investigates the effects of drugs on decision processes; and optogenetics, which uses light to control neurons for studying behavior.

What is a bias in decision making ?

Inherent bias in judgement and decision making that prevents us from being objective

Examples: When teachers reviewed papers they tended to give higher grades if the work was neat.

As reviewers when we review papers for publication, if the paper is from a well known famous lab we

are less likely to critique it. (blind review)

In sports cheers from a crowd can influence the decisions of the officials

Bias conveyed by brands and labels.

Concepts of anchoring effect, hindsight bias, confirmation bias, belief persistence, framing effect with examples.

Can our judgment be influenced by information that we know is irrelevant?

If a person is exposed to a number that should obviously not bias their decisions.

Real estate: Individuals will bias their estimate of how much a place is worth by fake information that is provided, without paying attention to the actual details of the place and objectively deciding how much the place should really be worth.

Sale: If the original price of an item is $150 and it says that it is being sold for 50% off we think it to be a real bargain, while the actual price of the item might have been $75

Anchoring effect changes our estimate of the value of an item and how many items we need.

(5 items for $2)

What is a heuristic? Why do we have heuristics ?

A heuristic is a mental shortcut that simplifies decision-making by allowing individuals to solve problems and make judgments quickly. Heuristics help reduce cognitive load and enhance efficiency in processing information, especially in complex situations where quick decisions are necessary.

What was the perceptual decision making task such as a random dot motion task ? What was the task structure ? What area of the brain was recorded from while the monkeys were performing the task?

Monkeys have to fixate on a fixation point

They have to judge the coherent

motion of random dots

Monkeys have to make a saccade

indicating direction of motion

Coherence of motion will make the

task harder or easier.

What technique was used to measure the activity from the brain? What was the main finding with respect to the coherence of dot motion…or in other words what did the neuronal firing tell the scientists about the coherence of dot motion

To examine the role of brain circuits in decision making–Recording of neural responses from

Lateral Intraparietal (LIP) cortex in monkeys (Electrophysiology) (glass electrode)

Probability of neuronal firing or neuronal response was higher when coherence

was high

(If direction of motion was unambiguous neuronal response was more robust )

The random dot motion experiments helped define the idea of decision variable and decision rule. What are the decision variable and rule ?

Decision variable which is the representation of evidence at the level of neurons

Neuronal response

More firing of a specific type of neuron influences the behavioral responses

A simple model for decision making in the brain

Decision rule - yes/no, more/less or category A/B

What are the two models of evidence accumulation in decision making –essentially explain drift diffusion and competing accumulator model. Concept of neuronal firing reaching a threshold as a mechanism of evidence accumulation in decision making.

•Evidence is accumulated over time into a decision variable (DV)-The diffusion model assumes that

decisions are made by a noisy process that accumulates information over time from a starting point toward one of two response criteria

•There are two decision bounds associated with the two choices (left or right)

•If DV reaches the upper bound first, choice 1 (dots moving left) is made; alternatively, if DV reaches the

lower bound first, choice 2 is made (dots moving right);

•The time between the start and end of accumulation is the decision time

•Rate of accumulation of information is called the drift rate

This idea is attractive because it is a single mechanism that can account for both which decision is made and how much time (or how many samples) it takes to commit to an answer

Value base decisions vs perceptual decisions

Many decisions we encounter do not involve evidence about the state of a percept or a proposition but require a choice between options that differ in their potential value, for example, deciding which shoes to purchase or which snack to choose.

Role of memory in value based decision making 1. Experiment with the novel food combinations 2. Experiment where spatial location of the snack was varied

Individuals are exposed to common foods and then 13 different novel food combinations,

were presented to the participants along with their names:

tea-jelly, tomato-jam,

popcorn-jelly beans,

beetroot-custard,

onion-mints,

pea-mousse,

olive-strawberry,

pesto-nutella,

spinach-pineapple smoothie,

raspberry-avocado smoothie,

vanilla-salt,

yogurt-pretzels

coffee-yogurt.

Participants learn a series of associations between snacks and a spatial location on the

screen.

• Some associations are trained twice as often as others, creating memories that are relatively

strong or weak.

• Participants are then probed to make choices between two locations, choices that require

retrieval of the memory for the location-snack association.

More direct test of the link between memory retrieval, evidence accumulation, and value-

based decisions. (value-based decisions that depended on associative memory between a

valued snack item and a spatial location on the screen)

Brain area: Hippocampus and ventro medial pre frontal cortex (vmPFC)

What did we learn from these experiments? (Decision-making)

In all these tasks, decisions about novel choice

options depend on the integration of past

memories.

What brain areas are associated with individuals performing a value based task.

Hippocampus and ventro medial pre frontal cortex (vmPFC)

Decisions about novel choices depend on integration of past memories.

Individuals who have anterograde amnesia where they cannot form new memories have difficulty imagining future events and learning to predict rewarding outcome

Perceptual decision vs Value-based decision

Perceptual decisions rely on evidence about perceptions, while value-based decisions focus on choosing between options based on their potential value.

Building on past events to make predictions about possible future outcomes is, what

memory is for. Emerging evidence indicates that memory plays an essential role in

many kinds of value-based decisions, particularly those that rely on the integration of

information across distinct past events or those that depend on prospection about

multi-step events leading to outcomes

Perceptual decision making : Perception of visual stimuli influence our

decisions

Ex: what direction is the car moving in?

what the color of the sky imply?

what does upturned lips (smile) on the face imply?

Brain areas involved in social decision making at the farmers market (fMRI) 1. Collect evidence of social context

sensory cortex, inferior temporal cortex, hippocampus

Brain areas involved in social decision making at the farmers market (fMRI): Inferring the mental state of others

anterior cingulate cortex

Brain areas involved in social decision making at the farmers market (fMRI): Value attached to each item . How much are you willing to pay for item?

Ventromedial prefrontal cortex (vmPFC), orbitofrontal cortex (OFC)

Brain areas involved in social decision making at the farmers market (fMRI): Updating of decision variable.

Lateral intra parietal cortex (LIP)

Brain areas involved in social decision making at the farmers market (fMRI): Motor action: purchase the item

motor cortex

Why study animal models?

Being social is conserved also all social species that live in groups.

Animal models allow performing electrophysiology, so we can measure neuronal activity with

millisecond precision.

fMRI is slow so we lose the temporal resolution, since neuronal communication happens at the

millisecond resolution

In human studies its hard to determine a value is assigned solely because of the value or is there a

contribution of other variables.

What is the state of mind hypothesis ?

Identifying neurons that reflect another individual’s covert intentions or

‘‘state of mind’’ has been a long sought goal in neuroscience and a central

proposed tenet of social decision making

What is the Iterated Prisoner’s Dilemma? Implementation of this game allows us to study temporal discounting and social decision making. Elaborate. For example what is temporal discounting ? How does temporal discounting affect our decision making ? Which species exhibit temporal discounting ?

The Iterated Prisoner’s Dilemma is a game used to study temporal discounting and social decision-making over multiple interactions.

The prisoner's dilemma is a paradox in decision analysis in which

two individuals acting in their own self-interests do not produce

the optimal outcome. People have developed many methods of

overcoming prisoner's dilemmas to choose better collective results

despite unfavorable individual incentives.

• The prisoner's dilemma presents a situation where two parties,

separated and unable to communicate, must each choose

between cooperating with the other or not.

• Two prisoners are accused of a crime. If one confesses and the other

does not, the one who confesses will be released immediately and the

other will spend 20 years in prison. If neither confesses, each will be

held only a few months. If both confess, they will each be jailed 15

years.

Temporal Discounting: In such a framework, each leave/stay decision is regarded as a choice between a

smaller-sooner stay reward and larger-later leave reward .

Temporal discounting, the tendency to devalue future rewards compared to immediate ones, has been observed in a wide range of species, including humans, rats, mice, pigeons, monkeys, and even some insects. 

Details about the experiments where the monkeys play the prisoners dilemma game. Make sure you know the task structure, what as the main result and what the neuronal recordings showed.

1. Monkeys were less likely to cooperate

overall and were more likely to select defection over

cooperation.

2. Moreover, they were less likely to reciprocate

cooperation following mutual cooperation when playing a

computer opponent, therefore leading to less mutually

beneficial interactions.

(social component involved)

3. Monkeys were almost twice as likely to cooperate

if they both cooperated on the previous trial, indicating

an intention to reciprocate mutual cooperation.

Neuronal responses in the anterior cingulate cortex during Prisoners Dilemma task provide information about the other’s covert intentions

or state of mind.

Task structure and main result from the monkeys pay per view paradigm

The "Pay-Per-View" paradigm is an experimental setup used to study value-based decision making in non-human primates. The task is designed to measure how much effort or cost a monkey is willing to endure to obtain visual stimuli, often social or rewarding images.

1. Trial Setup:

   • Monkeys are presented with a choice between two options:

     • A costly option that provides access to a specific image (e.g., social stimuli such as faces of dominant or subordinate monkeys).

     • A free or less costly option that provides either a neutral or less preferred image.

2. Cost Implementation:

   • The cost is usually measured in water rewards, delays, or effort.

   • Monkeys may have to sacrifice part of their water intake to view a preferred image.

   • Alternatively, they might have to endure a longer waiting time or perform more effortful actions.

3. Measurement:

   • The experiment records how frequently monkeys choose the costly option.

   • Researchers analyze preferences based on image type (e.g., social rank of faces, neutral vs. sexual images, familiar vs. unfamiliar faces, etc.).

4. Monkeys assign subjective value to images:

   • They willingly sacrifice rewards (e.g., water) to view socially relevant images, such as high-ranking individuals or potential mates.

   • Less interest is shown for neutral or non-social images.

5. Social hierarchy influences choices:

   • Monkeys prefer looking at dominant individuals but often avoid direct eye contact.

   • They may avoid looking at images of subordinate monkeys, reflecting real-world social behavior.

6. Neural Basis of Value-Based Viewing:

   • Studies have linked prefrontal and reward-related brain regions (orbitofrontal cortex, amygdala) to the decision-making process.

   • Dopaminergic signaling is thought to be involved in assigning value to social stimuli.

7. Comparison to Human Behavior:

   • Similar to humans, monkeys show willingness to pay for social information (e.g., humans pay for social media or exclusive content).

   • The paradigm provides insight into neuroeconomic and evolutionary aspects of social decision-making.

How does consumer psychology affect decision making ? What do you think defines our preferences and likes and dislikes ? How do advertisers use this information ?

Consumer psychology is influenced by

• affective processing: behavior that is quick pertains to current emotional

status, processing visual cues based on previous memory (“gut feeling”)

• subjective value

• persuasion

• attention

Digital and mobile advances in tracking marketplace activity have opened up a

large source of data, and in combination with analytics that can reveal not only

patterns of behavior, but also examine how small changes in marketing actions

affect them.

• These methods are used by companies to track consumer preferences

• They are also a tremendous source of information.

At the same time there is a certain ‘black box’ approach to this — modeling

behavior independent of attempting to understand the underlying processes.

What is the pepsi paradox ? What brain areas were involved in mediating this paradox ?

In blind taste tests subjects tend to prefer Pepsi over Coke or have no reliable

preference for one cola over the other, but yet Coke consistently outsells Pepsi.

• The paradox is that people exhibit a reliable preference for Coke when brand

information is available (e.g. in the supermarket), but no reliable preference for

Coke when no brand information is available (e.g. in blind taste tests).

• To demonstrate the powerful effects of cultural and social influences on the brain a

laboratory version of the Pepsi Challenge was conducted , where it was shown that

knowledge of the brand biased behavioral preferences away from Pepsi and in

favor of Coca-Cola.

At the neural level, behavioral preferences were found to be correlated with

activity in ventromedial prefrontal cortex (vmPFC) , and that furthermore

damage to this region abolished the biasing effects of brands

Scientists used the SHOP paradigm to study subjects behavior and preferences with respect to a product and willingness to spend. What brain did they find were important. (nucleus accumbens, medial prefrontal cortex, insula)

They found a specific sequence in which brain areas were active

1. Nucleus accumbens - activity increase

during product preference and price periods

Nacc is involved in anticipated positive arousal

such as financial gain

2. Medial Prefrontal cortex - mPFC activation

during the price period

mPFC activation correlates with actual

outcome such as positive event. Also active

during updating of decision variable

3. Insula - activity changes with decision to

purchase during the choice period

Insula is associated with anticipation of a

loss and an

aversion to financial risk

What is MVT? Is it conserved across species ?

This has been studied extensively and a marginal value theorem (MVT) of

behavioral ecology has been proposed.

MVT predicts that animals should leave the current patch when the energy

intake rate within the patch diminishes to the average energy-harvesting rate

in the environment.

Thus, the time that animals spend within a patch (i.e., patch-residence time)

depends upon a variety of factors, including the value of the current patch (in

terms of the resource being consumed), the value of other patches in the

environment, and the time it would take to travel to the next closest patch (i.e.,

travel time).

MVT hypothesis is conserved across species

Organisms as diverse as worms, bees, wasps, spiders, fish,

birds, seals and even plants obey the MVT. Evidence

demonstrates that human foragers and monkeys also obey the

predictions of the MVT in their hunting. The generality of the

MVT solution to the patch-leaving problem suggests that the

underlying mechanism is fundamental to the way organisms

make decisions.

What is temporal discounting ?

Temporal discounting is the tendency to devalue rewards or outcomes as they are delayed in time. In other words, people (and animals) prefer smaller, immediate rewards over larger, delayed rewards, even if waiting would result in a better outcome.

Patch leaving task in monkeys to study foraging behavior and how that relates to temporal discounting. What did the neural activity in dorsal cingulate cortex show during the patch leaving task.

Choice of blue rectangle (stay in patch) yields a

short delay (0.4 s, handling time) and reward

whose value diminishes by 20 μl per trial.

Choice of gray rectangle (leave patch) yields no

immediate reward and a long delay (travel time)

whose duration is indicated by the height of the

bar, and ultimately gives a reward of 306 μl.

Can the monkeys’ foraging behavior be explained in a delay-discounting or

temporal discounting framework.?

In such a framework, each leave/stay decision is regarded as a choice between a

smaller-sooner stay reward and larger-later leave reward .

• Such a decision model would naturally account for the monkeys’ observed

tendency to stay longer in patches when faced with longer travel times, as the

delay associated with patch-leaving would lead to discounting of the larger-later

reward.

With a longer delay ( time) the value of the reward decreases.

What other species did they study patch leaving behavior in ?

mice

Mice were trained to run on the track.

In the initial phase of training, mice were

provided with water whenever they approached

either the reward zone or poked into the water

port.

After consuming 20–40 rewards, mice were

conditioned to obtain water by running to the

opposite end of the platform, the trigger zone,

and running back to collect the reward.

Entry into the trigger zone activated an auditory

cue, which signaled availability of water at the

reward zone. Once mice performed about 60

runs they introduced different reward sizes

(small: 2–4 μl, large: 6–12 μl) signaled by distinct

auditory cues (mixture of high frequency and low

frequency tones, 0.1 s duration).

Large rewards required the mice to wait for the

correct tone before running to the reward

zone

Selection of egg-laying site in fruit flies

(Drosophila melanogaster) to probe the neural

circuit that governs simple decision-making

processes.

• First, Drosophila females pursue active

probing of the environment before depositing

each egg, apparently to evaluate site quality

for every egg.

• Second, Drosophila females can either accept

or reject a sucrose-containing medium,

depending on the context. Evolutionarily the

selection of the sucrose medium could have

been dependent on predation avoidance and

larval dietary balance (protein/carbohydrate

ratio).

• This value of the medium (decision

variable) is computed by insulin like

peptide 7 (IPL-7) neurons.

• These findings suggest that selection of egg-

laying site involves a simple decision-making

process in flies and provides a model system

to systematically dissect the decision making

process.

What is the advantage of studying the rodent brain ? What area of the rodent brain is analogous to primate and human brain when it comes to decision making

Important questions that have been addressed using rodent

models are

• How is the timescale for evidence integration determined and

implemented?

• What neural circuits support optimal integration of multisensory

information?

• How do the salience and value of decisions interact?

• How are biases based on previous experience incorporated into

developing decisions?

Many laboratories are finding it increasingly advantageous to seek some of

those answers in rodents. Indeed, in terms of perceptual decisions, the

cognitive capabilities of rodents are far from trivial. For instance, rats

combine multisensory information in a manner that approximates statistical

optimality, just as humans do. Moreover, they can accumulate information

over time to make decisions based on an abstract quantity.

A cortical area of interest in the study of decision-making, in the context of

updating the decision variable is the posterior parietal cortex (PPC)

PPC is analogous to area LIP in monkeys and humans.

• First, in rodents as in primates, PPC lies at the heart of a network of sensory

areas, receiving inputs from auditory, visual and somatosensory area; it is

therefore ideal for integration of multisensory inputs.

• Second, PPC neurons carry signals related to navigation and working

memory.

• Third, the activity of PPC neurons gradually increases during accumulation

of sensory information in decision-making tasks

What is a go/no-go task, two choice alternate task, two alternate forced choice with examples.

3. Many simple task designs in rodents can be used to study perceptual decision making.

This can be easily coupled to invasive brain imaging techniques.

Go/no-go task

(go- get out of the shower and grab the phone

no-go – stay in shower)

• The simplest kind of design to probe perceptual

decisions involves a ‘go/no-go’ task. In such a

task, the mice report the presence of a stimulus

attribute (for example, is it present or is it

vertical) by performing or withholding a single

action (for example, release a bar or lick a spout).

This design is often used in rodent studies,

because rodents learn it quickly.

• Problem with this task structure- Is the animal

responding less due to a lack of motivation or

drive?

A partial solution to the simple go/no-go

task is a task in which the subject is

given two choices of response,

depending on whether or not a stimulus

attribute is present. This design was

originally called “yes/no” but is

increasingly called “two-alternative

choice” (because of the choice between

two responses). It is often applied to

rodents, preferably with a symmetrical

apparatus wherein the animal indicates

the two choices in similar ways.

For example mice have to do a nose

poke in either the left or right port to

receive a reward.

Experiments done in my lab to study the role of sensory processing in decision making and how that is Disrupted in mouse models of Fragile X Syndrome.

Linking sensory discrimination to learning and decision making

Learning and cognition

Rodent behavior

Network mechanisms

Fragile X Syndrome

Can atypical sensory discrimination lead to deficits in behavior such as neurodevelopmental disorders

Role of dopamine in reward prediction errors

The difference between the actual outcome of a situation or action and the

expected outcome is the reward prediction error (RPE). A positive RPE

indicates the outcome was better than expected while a negative RPE

indicates it was worse than expected; the RPE is zero when events occur

according to expectations.

The difference between the actual outcome of a situation or action and the

expected outcome is the reward prediction error (RPE). A positive RPE

indicates the outcome was better than expected while a negative RPE

indicates it was worse than expected; the RPE is zero when events occur

according to expectations.

Neural Mechanisms of the credit assignment problem

Suppose you have to make a bunch of decisions in sequence without knowing the outcome of each then

learning has to happen in real time.

When a sequence of decisions results in reward how do we know which of the choices influenced the final

outcome ?

(wake up, maybe late, go for a run, eat breakfast, decide to not stop for coffee, arrive to work on time)

Reward prediction error

Temporal credit assignment problem

The idea here is that the brain is implementing a algorithm which is that we compare the expected reward for

making a particular choice with the actual reward you get and change your expected reward so that a better

decision can be made.

Then an update is made to the value network to adjust the expected reward for the future.

After enough exploration of the possibilities the algorithm converges to an optimal series of decisions that results

in the best possible outcome.

This idea is being extensively tested and implemented in optimal problem solving by

AI