Lecture 16 Decision Making - Sensory and Motor Neuro (Module 3)

Why is the decision-making saccade experimental setup, which looks at LIP, different than the setup for MT? For which brain region is the length of stimulus duration/appearance dependent on reaction time, and for which is it appearing for a fixed duration of time? Why might the difference in these designs be useful?

1. the monkey has to look at the motion stimulus for hundreds of milliseconds in order to judge what direction it is going (since the coherence is very low, and the motion stimulus is very noisy) - the judgment happens slowly

• in in the classic MT experiments (ex. Newsome) the monkey sees a moving dot stimulus shown for a SET amount of time - so the length of stimulus appearance is NOT dependent on how long the monkey takes to make a choice (reaction time)

• in the LIP experiment, the stimulus duration is NOT fixed - it is dependent on the monkey’s reaction time 

2. now the task is based on reaction time, because we want the monkey to make a saccade as soon as it arrives to a decision in order to cleanly separate the neural activity of the decision process (accumulating evidence towards ONE target, represented by a decision boundary) from the motor execution neural signal (the burst that triggers the eye movement/saccade) 

3. another difference has to do with the stimulus and target layout:

• MT: dot stimulus is inside the MT neuron’s receptive field —> lets us ask “is the motion in my neuron’s preferred direction, or not?”

• LIP: instead of the dot stimulus, one of the choice targets is located within that neuron’s receptive field, and the monkey just moves its eyes to the target that is IN its receptive field or NOT

If monkey moves its eyes to in-RF choice target: firing rate spikes the more evidence that the LIP neuron has (that the motion is going in the direction of this target)

If monkey moves its eyes to out-of-RF choice target: neuron barely fires (because that target is obviously outside of its receptive field) —> no ramp, no buildup to a decision boundary

What is the difference between what MT neurons represent, and what LIP neurons represent? How does the experimental setup change to reflect this?

• MT neurons represent motion direction (since dot stimulus is located within receptive field)

• LIP neurons represent decision towards a specific spatial target (either located in or out of receptive field)

What does the typical monkey behavioral data obtained during the reaction time motion discrimination task look like? Talk about this for both percent correct and mean RT (in milliseconds).

1. motion coherence/strength (x-axis) versus percent correct (y-axis)

• we see an upward slope for an S-shaped curve - which makes sense: the percentage of correct decisions increases with higher motion coherence

2. motion coherence (strength) (x-axis) versus mean reaction time (y-axis)

• we see a downward slope for an S-shaped curve - which makes sense. Lower motion coherences produce higher reaction times, because the monkey NEEDS that time to accumulate evidence towards a decision boundary

The monkey takes 400 milliseconds longer to decide direction of motion at low versus high coherence

Why is it useful for the monkey to integrate over a longer period of time at low coherence?

There is an observable difference across coherences (explaining the upward trend between motion coherence and reaction time) because of the increase in the amount of time needed to make a decision

• monkey spends half a second longer looking at the stimulus

What is the difference in the shape/appearance of neural activity between MT and LIP for graphs with time (x-axis) versus instantaneous OR integrated motion energy (y-axis)? Assume the task has to do with discriminating between leftward and rightward motion, with a weak rightward motion stimulus (3% coherence).

Model MT:

• instantaneous motion energy: "what direction is the dot stimulus moving in right now?”

• based on moment to moment signals in the receptive field: if the dots MOMENTARILY drift right, neurons that prefer rightward motion will fire. if the dots MOMENTARILY drift left, neurons that prefer leftward motion will fire.

• since coherence is low, firing rates fluctuate WILDLY

• no memory = no accumulation (of evidence)

Model LIP:

• integrated motion energy (integral of motion trace): “if I ADD up all the motion signals over time - with leftward motion being negative and rightward motion being positive - then what direction ends up WINNING OUT?”

• if we DO have memory and keep adding up the signals, the firing rate ends up creating a RAMP towards ONE choice target

• once it hits a threshold (decision boundary), this triggers the experimentally-contrived saccade to take place (where the monkey makes the eye movement to indicate its choice)

• for this motion stimulus, over time the integral will become overwhelmingly positive (but it would take a long time for a low-coherence stimulus - so the longer that you integrate over time, the better your estimate of whether motion was leftward or rightward)

Why would NOT waiting a long time to make a decision about the direction of motion lead to inaccurate results, when thinking about integrated motion area/what LIP neural activity looks like? Let’s say we have a weak rightward motion stimulus at around 3% coherence, and negative y-values represent leftward motion (for MT, and for LIP, it is the sum of MT instantaneous motion signals) while positive y-values represent rightward motion.

• Remember that MT fluctuates wildly because is neurons are just responding to the stimulus on a moment-to-moment basis - NOT accumulating evidence or storing memory

• If a few dots move leftward, that triggers the leftward preferring neurons to fire and we see a sharp dip towards negative y-values (leftward motion) on the graph

• when LIP adds these initial signals up, we see dips towards negative values on its graph (since the sum of the MT signals so far is pointing towards leftward motion)

• but if we wait, we will end up seeing that this goes AGAINST the general trend of the LIP integral, which generally supports rightward motion (is heading in a positive direction) - so that means it is best to wait somewhat longer before a decision is made

Talk about a graph representing the average activity of 54 LIP neurons, where solid curves represent motion towards the target within the response field (Tin) while dashed curves represent motion towards the opposite target (Tout) - how does firing differ between the two? How does the slope of the line look when coherence is high (versus low)? Why do all the solid lines (Tin curves) come together before a saccade is made? 

Firing rates differ between the two because each particular LIP neuron prefers the target within its response field and will only really fire when the monkey makes a saccade in the direction of the Tin target (NOT Tout)

• we see the ramp of accumulated evidence towards choosing Tin (high response)

• we see a low, flat line when evidence supports choosing Tout (again, just for the particular neuron being recorded - a different neuron might fire a lot)

The Tin curves converge (come together) right before the eye movement because the brain uses a FIXED FIRING-RATE THRESHOLD to initiate a saccade 

• so the curves all have to hit the same activity level (collapse onto each other) before the eye moves, no matter how different they might look during the evidence accumulation stage

• this is the trigger for the animal to indicate its choice (again, experimentally contrived linkage of perception (decision) to action (saccade))

NOTE: responses are also aligned at the onset of the motion stimulus (we see a small peak that looks similar for all the curves)

Why can you align the trials to stimulus onset (appearance) OR saccade onset, but not both? What does this have to do with the design of the LIP/decision-making task?

The monkey’s reaction time is different on every single trial (sometimes it decides early and the saccade happens faster, and sometimes it decides slow and the saccade happens late)

• if you choose to align the trials on stimulus onset, then the saccade ends up happening at different times

• if you align the trials on saccade initiation, then the activity that happens when you show the stimulus will no longer be aligned

What does LIP activity represent? How can we define this term? What model represents the means of MT instantaneous motion energy as bell curves, and represents a LIP integral using two decision boundaries (for the two directions)? Assume we have two motion stimuli (A and B) at two different coherences - A is higher, B is lower.

Decision variable: continuous index (represented by a gradually changing line) of how likely one decision (ex. a saccade to the right) is relative to the other possible decision (a saccade to the left). Represents evidence accumulation

MT graph (momentary evidence in MT):

• evidence for right versus left on x-axis (with positive values representing the right and negative x-values representing the left - 0 is used to separate the two)

• we have two bell curves representing the motion stimuli (A and B): since A = higher coherence motion stimulus, its bell curve’s mean (peak) is shifted towards the positive x-values

• B’s curve is shifted more towards 0 or the negative x-values (left), because its coherence isn’t as strong and so the direction of motion isn’t as obvious

LIP graph (integral/summed activity from MT), which represents the drift diffusion model that has two decision boundaries:

• + represents right decision boundary, - represents left decision boundary. 0 lies between them.

• for A, the cumulative (added) stimulus activity from MT creates a signal that hits the rightward motion decision boundary MORE QUICKLY, which triggers the rightward choice. This makes sense since (again) A has a higher coherence/stronger motion signal)

• for B, the activity still hits the rightward decision boundary, but it takes longer (because the stimulus is lower coherence)

Why does the drift diffusion model naturally explain why reaction times become smaller and percent correct becomes greater? 

High coherence stimulus (strong motion signal): 

• MT more likely to fluctuate towards rightward motion (rightward motion neurons fire), even moment-to-moment

• LIP will take those rightward motion signals and sum them up —> the integral or decision variable hits the decision boundary sooner

What is the difference between the decision variable and decision boundary?

Decision variable: continuous index (represented by a gradually changing line) of how likely one decision (ex. a saccade to the right) is relative to the other possible decision (a saccade to the left). Represents evidence accumulation

Decision boundary: static (a fixed threshold that the decision variable has to reach in order to trigger the decision)

How does the slope of the decision variable or boundary predict both the choice of the monkey and the timing of the choice (reaction time), when we plot time versus firing rate?

positive slope: high firing rate, indicates that monkey made choice towards the target within the receptive field of that neuron

negative slope: lower firing rate, indicates that the monkey made its choice towards the target OUT of that neuron’s receptive field

steep slope: monkey took less time to make its choice (evidence accumulated and reached decision boundary faster)

gradual slope: monkey took less time to make its choice (so evidence took more time to accumulate and decision boundary was reached slower)

Why is the Shadlen model of decision-making (drift diffusion) important?

It’s a specific, concrete proposal for how sensory signals (direction of motion) are tuned into a binary decision (to move eyes to one of two possible targets) 

The Shadlen (drift-diffusion) model explains many experimental observations, but what is an example of a study whose results contradicted this model? What does this demonstrates

Katz et al. —> found that reversibly inactivating LIP didn’t affect performance of the direction discrimination task

• so is LIP really summing up the fluctuations/neural activity of MT to create an integral/decision variable that hits decision boundaries, or is its activity (firing rate of its neurons) just CORRELATED with that of MT?

• DEMONSTRATES THAT correlation ≠ causation

Sum up what the Shadlen or drift diffusion model predicts.

That the neural representation of the decision to make a saccade in the motor task builds up GRADUALLY (evidence accumulation). Eventually, the decision variable reaches a bound to trigger the action (saccade).

The drift-diffusion model, by virtue of saying that LIP accumulates evidence (through summing up MT neural signals) to make a decision about a certain stimulus feature, suggests that the decision is partially formed BEFORE the bound is reached (since we see a “ramping” of neural activity towards a bound). What does this imply?

That artificially triggering the action early should reveal a trace of the developing decision in the oculomotor circuitry.

• basically, if we force a saccade early, BEFORE LIP activity actually reaches the decision boundary….

• ….then (because LIP activity encodes the accumulated evidence UP TO THAT POINT), we should see the saccade be BIASED towards the choice supported by the current LIP ramp

Gold and Shadlen wanted to force the saccade to occur early to see if the accumulated evidence (encoded by LIP activity) would BIAS the saccade in favor of where the “ramp” was going (the direction of the decision variable). What did they do in order to carry this out, and what were their results? Did we see interaction of the microstim with a partially formed plan (of LIP)?

• they used the same direction discrimination task (with trial duration depending on reaction time)

• BUT they microstimulated the FEF to evoke a saccade SHORTLY BEFORE the monkey was ready to make its decision

• the microstimulation involved evoking a saccade orthogonal to the axis defined by one of the choice targets - but what neurons were being microstimulated doesn’t really matter. What IS important is what choice target axis the saccade ended up being BIASED TOWARDS

• if the saccade ended up being biased towards an upwards choice target, that shows that LIP activity up to that point was “ramping up” towards the upwards decision boundary (saccade evoked by FEF stimulation was PARTIALLY biased to land in direction specified by random dot stimulus)

• NOTE: it couldn’t be fully or perfectly biased, because of the microstim (injected rightward signal, for instance) was still influencing where the saccade went 

So, in summary, the microstimulation (ex. an electrically injected rightward signal) DID interact with the partially formed plan to make an up/down saccade based on the motion stimulus

What two factors was the deviation (bias) of the saccade towards the random dot stimulus directions influenced by (in the Gold and Shadlen microstimulation experiments)? What does this suggest?

• higher motion coherences (stronger motion signals)

• longer viewing durations

BOTH are consistent with the effect of a growing decision variable in LIP activity.

• this suggests that the neural correlate of the decision (i.e. neurons that fire around the time that the decision is made) develops within the oculomotor circuitry over the course of the trial…

• as opposed to there being a SEPARATE abstract decision-making center that only signals when the decision has been made (such that LIP activity would only be correlated to the choice, instead of there being strong evidence in favor of causation)

If LIP were only correlated (instead of being responsible or partially responsible for the monkey’s choice) we would just see a sharp jump (in activity) around the time the monkey makes a choice, NOT smooth gradual ramping that relates to the motion evidence we have collected from MT so far.

After the monkey made an initial saccade, what did it then do to correct itself?

Make a saccade to correct itself (so instead of remaining fixated at some point diagonal to the original fixation point - which was located in-between the choice targets - it would then “jump” to either the up or the down targets from there)

NOTE: obviously we only cared about the initial saccade, since its direction what the microstimulation of FEF was interacting with (and revealing the bias of)

On a graph of viewing duration (in milliseconds) versus amount of deviation (in degrees) of initial saccade relative to the original fixation point, what would we expect to see, in relation to the Gold and Shadlen microstimulation experiments?

We would expect to see steeper slopes (indicating more deviation at lower values for viewing duration/reaction time) for HIGHER coherences.

Can FEF be microstimulated at different times during the trial? What happens if it is microstimulated too early?

Yes. BUT we won’t see interaction with the plan to move the eyes (make a saccade), because no strong plan exists yet 

• so this will lead to no/very little effect taking place.

• microstimulation can’t bias a plan when there is no existing or strong plan yet!

Would you see a neural correlate of the decision in oculomotor circuitry (evidence accumulation) if the monkey did NOT know what the possible motor outputs would be? How did Gold and Shadlen modify the experimental set-up of the memory saccade task to test this?

No. 

Gold and Shadlen tested this with the following experimental setup (known as the colored-target task):

• monkey fixates on a point

• random dot stimulus appears at this fixation point, and exhibits motion

• colored targets switch on (ONLY at the very end - this is a difference from the other experimental setups)

• monkey makes a saccade to the target it thinks the random dot stimulus is moving towards

The two differences to the prior experiments include:

• the choice targets were two different colors (ex. red and green), and the monkey was trained to associate the two possible directions of motion with two different colors)

• this allowed for MAPPING (association of) motion judgments to color

Also:

• the trial duration was NOT based on the reaction time of the monkey…

• so the monkey couldn’t end the trial early (with a saccade) even if it knew what the direction of the stimulus was already, because the choice targets only appeared at the end of the trial

• so the monkey couldn’t plan the eye movement it wanted to make during the viewing of the motion stimulus

Even if the choice targets only appeared at the end, why couldn’t the monkey plan the saccade? It still might know the direction of motion before the targets even appear, so even if it can only make the saccade when the targets are present, the brain could technically still “imagine” a target there.

Until the targets actually appear, the monkey doesn’t enter the FINALL movement-preparation (saccade generation/initiation) stage.

• it’s basically the difference between imagining throwing a ball and actually having the ball in your hand, available to throw it…

• …or the intensity difference between a yellow and a green light

• so the ability of the oculomotor system to generate saccades depends on the presence of the choice targets, even if the monkey might decide on what the direction of the stimulus was early on.

Go more into detail about the results of the colored-target task that Gold and Shadlen did, which involved FEF microstimulation and ONLY showing the choice targets at the very end. What did they end up finding?

That the neural correlate (pattern of activity that tracks what decision the monkey is making, or the DECISION SIGNAL) of the decision process is ONLY seen in the oculomotor circuitry when the monkey KNOWS what possible eye movements it will have to make.

• so the interaction between the evolving decision variable and the microstimulation signals only happens when the monkey KNOWS what eye movements to make….

• …meaning that for this experimental set-up, when choice targets don’t appear at the start (only the end), decision signals only occur in a different part of the brain (more abstract decision-making center?)

• this is speculative

What do we see on a graph of viewing time (x-axis) and magnitude of duration (y-axis) with the colored-target task, where the choice targets were only shown at the very end?

• we see FLAT lines (that don’t change, or aren’t affected by, the amount of viewing time)

• when we have multiple different coherences plotted, nothing changes with that either

• shows that the decision variable is NOT encoded in (or coming from) LIP for this experiment —> so we don’t see a ramp (or change in slope) toward a decision boundary