Ph.D. Proposal on Primate Brain and Visual Decision-Making

A collection of vocabulary flashcards based on key concepts from Yueyue Sapphire Hou's Ph.D. proposal on how the primate brain uses visual information for learning and decision-making.

SLIDE 1

[1]

“Let’s begin with a simple observation“

• what’s truly remarkable

• how we SENSE the world

• how quickly our brains MAKE SENSE of it

• REACT in our own styles

“What’s truly remarkable is not just how we sense the world, but how quickly our brains make sense of it. And this allows us to react to the world in our own styles. “

SLIDE 2

[2]

• present my Ph.D. PROPOSAL:

• primate brain, visual information, learn & make decisions

SLIDE 3

[3]

“To tackle this, I focus on“

• non-human primate visual system [SIMILAR]

• >> half macaque brain DEVOTED visual processing

• closely mirrors brain ORGANIZATION

• IDEAL MODEL: what we see —> what we decide

“In fact, over half of the macaque monkey's brain is DEVOTED to visual processing“

“and it closely mirrors human brain organization“

“This makes macaques an ideal model for studying how we turn what we see into what we decide.”

SLIDE 4

[4]

“Inside the macaque visual system, circuits of neurons"

• constantly COMMUNICATE

• use ELECTRODES to listen and monitor

• feature: NOT respond the SAME WAY to the SAME STIMULUS

• VARY TRIAL-TO-TRIAL

“And one surprising feature of them is that they don't always respond the same way to the same stimulus."

“In other words, even when the visual input doesn't change, the response of a group of neurons can vary quite a bit from trial to trial.“

SLIDE 5

[5]

“To visualize this, here’s a raster plot.“

• describe: row, x-axis, tick-mark

• “By responding to the same stimulus multiple times“

• consistent INCREASE after stim onset

• VISIBLE VARIABILITY across trials

• trial-to-trial variability

“As you can see, by responding to the same stimulus multiple times, there's a consistent increase in spike firing after stimulus onset, but there's also visible variability across trials. This is what we call trial-to-trial variability.“

***SLIDE 5***

***[5]***

“And this is exactly where my proposal comes in:“

• ignore variability (dismissed as noise)

• focus on variability

• because it’s not random

• 1) shared across neurons; 2) tightly structured; 3) after training

• more than JUST passively respond to what’s OUT there

• internal principles: how we learn and make decisions (don’t fully understand yet)

“instead of ignoring this variability, which often is dismissed as noise, I focus on it. “

“Because this variability is not random. It’s often shared across neurons and tightly structured. Even after days of training, this variability is still there. “

“Indeed, this variability may suggest the brain is doing more than just passively responding to what’s out there. Its internal structure and principles may play a fundamental role in how we learn and make decisions, which we don’t truly understand yet.”

SLIDE 6

[6]

“So here’s the core idea behind my research:“

• I hypothesize that:

• “to test this”

• 2 metrics to study this kind of variability

• CP: fluctuation in neural activity correlates with behavioral choices [Aim 1]

• NC: how variability is shared across neurons [Aim 2]

“The first is choice probability, which captures how fluctuations in neural activity relate to behavioral choices. “

“The second is noise correlation, which measures how variability is shared across neurons.”

HGIH noise correlation = often is dismissed as limiting the amount of information the brain can encode.

“With that overview in place, let’s dive into Aim 1 about choice probability.” 

Slide 7

[7]

“A question I often get is what is choice probability.“

• metric: predict choice from “noise“

• noise: spikes/LFPs

• choice: preferred/anti- based on TUNING

• example: dots cloud, coherent right, to choose which direction. MEANWHILE, retinotopic site, recorded neurons FIRE more towards right. + behavioral outcome = choice

• 0<CP<1. =0.5 variability tells nothing about choice; >0.5, neurons fire more when the animal chooses the stimulus the neurons prefer; <0.5, fire LESS when choose neurons’ preference.

“It's a metric researchers developed to predict animal behavioral choices based on the ‘noise’ made by neurons. This noise can be attributed to spikes or LFP signals. And the choices are categorized as either preferred or anti-preferred based on the neurons' tuning to visual stimuli. For example, in our task, we show a dots cloud and among these dots, some of them are moving coherently to the right and the rest are moving at random. The monkeys' task is to choose which direction the dots are moving coherently. In this case, from the retinotopic site, the neurons we recorded prefer right rather than left, and by combining their tuning with behavioral outcome, we were able to quantify the monkeys' behavioral choices. “

“CP values range from 0 to 1. A value of 0.5 means the neuronal variability tells us nothing about the animal's choice. If it's above 0.5, that means the neurons are likely to fire more when the animal chooses the stimulus that the same neurons prefer. And if it’s below 0.5, it means neurons fire less when the animal chooses the neurons’ preference. “

***SLIDE 7***

***[7]***

“Although CP has been widely used in the field”

• gap in how to interpret CP [correlational nature]

• Are these noises made by the local neurons directly drive the decision, or are they only mimicking other neurons’ conversation, so eliminating these local neurons won’t truly affect decision-making?“

SLIDE 8

[8]

“To visualize why this distinction matters in the visual system,”

• walk through a simplified visual circuit [focus on area MT]

• MT receives motion signals [V1 & V3A]

• MT projects to downstream [MST = motor planning]

• in turn project to higher-order area [prefrontal]

• NOT one-way traffic

• FEEDBACK from upstream back into MT

• rationale: CP in area MT, how does MT contribute to decision?

• feedforward driving a choice/ correlated decision activity (or both)

“MT receives motion signals from earlier visual areas like V1 and V3A, and projects to downstream regions like MST, which are involved in motor planning. These, in turn, project to higher-order decision areas like the prefrontal cortex. But it’s not just one-way traffic—there’s also feedback from those upstream areas back into MT. So when we compute CP from area MT, we don't know [how does MT contribute to decision] as it could reflect a feedforward signal that's driving a choice—or a signal reflected other correlated decision-related activity. Or both.“

SLIDE 9

[9]

“To disentangle these possibilities, I took a causal approach.”

• use a dataset (spikes and LFP)

• while monkeys performed random dots

• *ideal to activates spikes in MT (lesion studies)

• temporarily inactivate spikes by muscimol

• * what happens to CP when area MT no longer sends feed-forward signals out

“To disentangle these possibilities, I took a causal approach. I used a dataset collected from MT, while monkeys performed a random dots motion discrimination task—a classic decision-making paradigm. This task is ideal to activates spikes in MT, which has been confirmed by lesion studies. Then, in some sessions, area MT was temporarily inactivated by injecting muscimol, a GABAa agonist that silences local spikes. This allowed me to ask: What happens to CP when area MT no longer sends feed forward signals out.“

SLIDE 10

[10]

“Just to recap and visualize our manipulation.”

• Pre: arrows intact

• Post: abolish 2 arrows from area MT

“Just to recap and visualize our manipulation. Pre-inactivation, we have all these arrows intact; and after inactivation, we abolished these two arrows originating from area MT. “

SLIDE 11

[11]

“Here comes the twist.“

• CP used from spikes

• inactivation, no spikes

• CP now from LFPs

• LFP = summed, rhythmic, coordinated activity

• another benefit: diff freqs = neural processing

1. time → freq (Fourier)

2. slide window → spectrogram (describe)

3. *similar to spikes, LFP respond to stim

4. Spectrogram per trial, categorize spectrogram into two choices

5. For each choice, at each bin, a distribution of LFP power across trials

6. signal detection theory, ROC, area

“So by computing CP from LFPs across different frequency bands, before and after inactivation, we can study the contributions of these feedforward and feedback pathways.”

SLIDE 12

[12]

“Here’s what we found“

• describe: before inactivation, freq, time, color = CP

• 0-70: not reach

• 70-270: robust CP in 2 freqs

• high gamma (70-150 Hz): peak early around 80 ms

• low gamma (25-50 Hz): peak slightly later around 120 ms

• decision-related signal in alpha-beta, but pre-stim

“From zero to 70 ms, the visual signal from stimuli has not reached area MT yet, but during the stimulus response period from 70 ms to 270 ms we saw robust CP in two key frequency bands during the stimulus response period: the high gamma range (70–150 Hz), peaking early around 80 ms. the low gamma range (25–50 Hz), peaking slightly later, around 120 ms. We also saw decision-related signals in alpha and beta bands, but interestingly, these occurred before the stimulus onset. I’m happy to discuss what we think it's happening in alpha-beta during Q&A.”

SLIDE 13

[13]

“Now let’s look at what happens after inactivation.”

• when spikes in MT were eliminated

• *key result, offer clues, dissociate, feedforward/feedback contributions

• high gamma CP disappeared: reflect a causal+feedforward contribution to decision

• low gamma CP remained, even stronger during delay (no visual stim)

• suggest: other than feedforward drive, possible feedback from downstream areas

“when local spiking in MT was silenced. And this is the main result that offers us clues on dissociating feedforward and feedback contributions. The high gamma CP signal disappeared. Gone. This strongly suggests that high gamma CP reflects a causal, feedforward contribution from MT to the decision. But CP in the low gamma band persisted, and even became stronger during the post-stimulus delay period, when no visual stimulus was on the screen. This suggests that low gamma CP reflects something other than feedforward drive—possibly feedback from downstream areas. “

SLIDE 14

[14]

“To map these results to understand the mechanisms,“

• (orange arrow): disappearance of high gamma CP after inactivation

• supports “MT causally drives animal’s decision“

• (blue arrow): persistence of low gamma CP

• reflects “shared noise correlations btw MT and other areas like V3A and MST”

• some interactions = feedforward in nature

• favor feedback origin cuz low gamma CP peaks later after inactivation

• temporal shift aligns with previous work

• “working memory influences low gamma oscillations“

“To map these results to understand the mechanisms, represented by orange arrows, the disappearance of high gamma CP after inactivation supports that area MT causally drives the animal's decision. Represented by blue arrows, the persistence of low gamma CP reflects shared noise correlations between neural activity in MT and other areas such as V3A and MST. Some of these interactions could still be feedforward in nature p, but evidence favors a feedback origin is the fact that low gamma CP peaks later. That temporal shift aligns with previous work showing that working memory influences low gamma oscillations.”

SLIDE 15

[15]

“Putting it all together, so what does this tell us?“

“By computing LFP-based CP with muscimol inactivation, we were able to isolate decision-related signals that cause decision from those that are correlated but not driving the decision. These findings open the door to future work on where these feedback signals originate, and how to use different frequency bands to study decision-making across brain areas.“

SLIDE 16

[16]

“So far, we’ve been talking about a classic task”

• dots discrimination task (great for MT)

• artificial, not how we learn, training

• rapid, biologically meaningful (recognize face, expression, see him a few times)

• few-shot learning (challenge to understand, Aim 2)

“—random dot motion discrimination—which is great for driving activity in area MT. But the problem is: these stimuli are artificial. They’re not how we learn naturally. And in practice, monkeys take months of training to learn them well.

In contrast, the kind of learning we perform every day—especially as humans—is often rapid and biologically meaningful. We can learn to recognize a familiar face—even in a different expression, even though we only see that person just a few times.

This capability is referred to as few-shot learning. And while it comes naturally to us, it remains a major challenge to understand how brains operate to give rise to this ability. So in Aim 2, I probe into the neural mechanisms of few-shot learning. “

SLIDE 17

[17]

“One candidate mechanism is noise correlation”

• DEF: shared trial-by-trial variability btw neurons

• record simultaneously, variability tends to fluctuate in same direction

• *traditional, a problem

• correlated, no cancel out, limit info, SNR harmed

• describe FIGURE

• “but here’s the catch“

• NC is ubiquitous: limitation vs. “what are they doing for the brain?“

“that is, shared trial-by-trial variability between neurons. In simpler terms, if recording from two neurons simultaneously, the variability in this neuron tends to fluctuate in the same direction as the other one. 

Traditionally, noise correlation has been viewed as a problem. Because they’re correlated, so averaging over trials cannot cancel them out, so it limits the amount of stimulus-dependent information the brain can extract. In a simpler term, signal-to-noise ratio is harmed by noise correlation. 

But here’s the catch: noise correlations are ubiquitous in the brain. So instead of seeing them as a limitation, maybe we should ask: What are they doing for the brain?”

SLIDE 18

[18]

“noise correlation plays a functional role in shaping learning speed”

• specifically, I hypothesize…

• testable, but not evaluated

“This is a testable hypothesis that hasn't been evaluated empirically.”

SLIDE 19

[19]

“To illustrate this idea, let me give you a quick analogy.”

• downtown Boston, Fenway, baseball

• run late, noise around, not help

• loud cheer from the stadium (high correlated noise—many people, all at once, from one place)

• SAVE TIME from wandering aimlessly

• “in the same way“ correlated neural noise guide to find task-relevant dimensions

• so behavioral performance happens more quickly

“Imagine I’m in downtown Boston, trying to find Fenway Park for a baseball game. I’m running late, there’s a lot of noise around me—street vendors, traffic, unrelated conversations. None of that helps.

But suddenly, I hear a loud cheer echoing from the stadium. It’s highly correlated noise—it comes from many people, all at once, and all watching the game in the stadium, which is the place I’m looking for. It saves me TIME from wandering aimlessly. 

In the same way, correlated neural noise might help guide the brain quickly toward task-relevant dimensions so behavioral performance happens more quickly. “

SLIDE 20

[20]

“To test this, I will borrow a well-designed delayed match-to-identity task.”

• sequential images (separate by delay)

• belong to same identity (ex. same person)

• identities defined across a variety, including real-word objects (encourage abstraction and generalization)

• 3 strengths: 1) naturalistic; 2) learning at stimulus level; 3) tease apart relevant/irre features (facial expression vs. overall facial structure)

“On each trial, the monkey sees two images presented sequentially, separated by a brief delay. Then, two saccade targets appear. The monkey's job is to report whether the two images belonged to the same identity--for example if they are the same person with different facial expressions. If  The categories are defined across a variety of stimuli, including real-world objects and artificial shapes, to encourage abstraction.

There're three main strengths of this design: 1) stimuli are naturalistic; 2) this task structure allows me to analyze learning at the stimulus level—tracking how quickly each image pair is learned over time; 3) the task helps tease apart relevant and irrelevant task-related features. Given the task design, if monkeys focus on facial expression, then its answers would be wrong in both cases, which is irrelevant to the task; but if monkeys focus on overall facial structure, then they will be able to quantify identity correctly. “

SLIDE 21

[21]

“During this task, I will record from both V4 and IT cortex by using multi-contact probes to record multi-unit activity. “

• ventral “WHAT“ pathway

• V4 tuned to interm features

• IT more categorical

• what + where

“Both these two areas along the ventral visual pathway. V4 is tuned to intermediate features—like curvature, orientation, and shape fragments. IT is more categorical, tuned to complex objects and identity-level information. This setup allows me to ask not only what regions are involved in learning, but where noise correlations matter most.”

SLIDE 22

[22]

“To study this computationally, we use two related concepts: “

• signal correlation: how similarly, 2 neurons, respond, on average, across stimuli

• describe FIGURES

“Signal correlation refers to how similarly two neurons respond on average across different stimuli. On the left, these two neurons are similarly tuned, because they tend to fire in the same direction in response to stimulus 1. On the right, these two neurons are more dissimilarly tuned. “

SLIDE 23

[23]

• Noise correlation: how similarly, fluctuate, trial-to-trial, one stim

• trial basis

• although similarly tuned, still variability for one stim

• variability correlate positively

• similarly tuned also negative noise corr

“Noise correlation refers to how similarly they fluctuate from trial to trial, even when the same stimulus is shown. So here we define on the trial basis. Although these two neurons are similarly tuned, but for each stimulus there’s variability, and these variabilities tend to correlate positively. On the other hand, similarly tuned neurons can also exhibit negatively noise correlations. “

SLIDE 24

“In my hypothesis, the scenario I suggest to induce few-shot learning”

• positive noise corre + similarly tuned

• simpler term: prefer the same, vary consistently

• what happen in my proposed task?

“is to have high noise correlation within similarly tuned neurons. In a simpler term: not only do these neurons prefer the same images, but they also tend to vary together from trial to trial in a consistent way. Now imagine what this could happen in my proposed task.” 

SLIDE 25

[25]

“It is a more complex scenario, as neurons’ job is to tell the two sequential images apart.“

• obvious, prefer stimuli and elicit positive noise corr

• *distribution of responses across trials are highly overlapped

• overlap: harder tell stimuli apart

• twist: among neurons, rely on whose info (noise in city)

• overlap helps downstream readout, to spot on (save time wandering), learn faster

• tradeoff: performance won’t be precise

• right neighborhood, no exact address

“Obviously, these two neurons both prefer the stimuli the same way and elicit positive noise correlations. So Their distribution of responses over trials creates a large overlap, so it becomes harder to tell stimuli apart. 

However, here’s what our hypothesis comes in. because this overlap is so obvious, imagine there are thousands of neurons shouting out their different opinions in the crowd. Then, relying on whose information becomes hard to decide, but it is possible that because this overlap is so distinct for downstream readout neurons to recognize, it may help the brain quickly spot on this overlap so save the time aimlessly wandering for clues so to learn the task faster. but the tradeoff would be the performance won’t be as precise. -- kind of like being directed to the right neighborhood, but you don’t have the exact address.”

SLIDE 26

[26]

“Here’s what I plan to analyze“

• treat electrode as neuron

• signal corr btw pairs of neurons

• each neuron: response vector (firing across all stimuli, averaged over repetitions)

• correlate vectors across neuron pairs (quantify how similarly they’re tuned, gives SC)

• for neuron pairs with high SC, NC separately for each stim (measure how much activity co-vary across trials, even same stim)

• predict: pair of sequentially presented images: high NC + similarly tuned —> learn more quickly (reach threshold fewer trials)

• tradeoff: final accuracy is lower than slower-learned image (neural code overlap, less distinct)

“I’ll treat each electrode as a single neuron and start by computing signal correlation between pairs of neurons.

For each neuron, I’ll construct a response vector that reflects its average firing rate across all stimuli, averaged over repetitions. By correlating these vectors across neuron pairs, I can quantify how similarly they are tuned—this gives me signal correlation.

Then, for neuron pairs with high signal correlation, I’ll compute their noise correlation separately for each stimulus. This measures how much their activity co-varies across trials, even when the same image is shown.

I predict that if a pair of sequentially presented images both elicit high noise correlation among similarly tuned neurons, then that pair should be learned more quickly—in other words, it will reach the behavioral accuracy threshold in fewer trials.

However, as a tradeoff for faster learning, I predict that the final accuracy may be lower than for slower-learned image pairs—since the neural code is more overlapping, and therefore less distinct.”

SLIDE 27

[27]

“Now, let’s consider two possibilities across brain areas“

• 1: this prediction is elicited only in IT cortex

• suggest: IT = key site, learning speed can be predicted by IT NC

• 2: this prediction is stronger in V4 than IT

• suggest: V4 structured high NC as scaffold for IT to form stable category representations

• expect: high NC in V4, but low NC in IT (IT has already shaped more refined format)

“Possibility 1: This prediction is elicited in IT cortex.
That would suggest that IT itself is the key site where high noise correlation supports rapid learning. In this case, we expect learning speed to be most strongly predicted by IT noise correlation.

Possibility 2: The effect is stronger in V4.
In this view, V4 provides structured, high-noise-correlation input that acts as a scaffold for IT to form stable category representations.
So we’d expect high noise correlation in V4—but lower correlation in IT, since IT has already shaped its output into a more refined, low-dimensional format”

SLIDE 28

[28]

“Now, what if we disrupt that structure?“

• musc, selectively silence either V4 or IT

• ask: “what happens to learning when structured variability is removed?“

• 1: IT is the key site, inactivate IT, impair (particular for stimuli that are learned faster)

• 2: V4 provides scaffold, inactivate V4, impair (IT still operate but with less discrepancy in learning speed)

Using muscimol inactivation, we can selectively silence activity in either V4 or IT. This allows us to ask:
What happens to learning when structured variability is removed?

• If IT is the key site for using noise correlation to support learning, then inactivating IT should impair learning—particularly for stimuli that rely on high IT correlation.

• But if V4 provides the scaffold, then inactivating V4 should selectively impair learning speed—while IT may still operate, but without the structured input that accelerates the process.“

SLIDE 29

[29]

“I ask whether shared variability among similarly tuned neurons helps the brain learn faster—not by improving encoding precision, but by constraining neural activity into task-relevant subspaces that are easier for downstream readouts to learn from.

If true, this would suggest that noise correlation is not just a limitation—but rather, a functional tool the brain uses to guide learning under conditions of limited experience.

Ultimately, this work could reshape how we think about neural variability—not as something to be averaged away, but as something the brain leverages to learn quickly and efficiently.“

SLIDE 30

[30]

““To wrap up:

This proposal asks a simple but powerful question:

How does the primate brain learn to make visual decisions, especially sometimes so quickly—often with just a few examples?

I approach this from two angles:\n

• Aim 1 shows that decision-related neural signals are not monolithic—some are causal, others correlated, and LFP frequency allows us to tease them apart.

• Aim 2 explores noise correlation can actually induce few-shot learning. 

In terms of the timeline, till this point, I have been wrapping up my findings for Aim 1 by drafting a manuscript. For Aim2, the data collection stage is about to start as my monkey is recovering from a neck problem. 

So the last but actually the first question I ask myself is why should we care about the ways in which the brain gives rise to visual decision-making? Well my answer to this lies in the profound implications it has. By studying the brain we hold the key to not only decoding brain functions but also improving technologies designed to mimic human intelligence.  

So with that in mind, thank you very much—and I’d be happy to take your questions“