1/29
Looks like no tags are added yet.
Why are two eyes useful for depth perception?
The two eyes sit in fixed positions about 6 cm apart, giving two slightly offset views of the scene that the brain can compare to compute depth
Both eyes see mostly the same scene, but from slightly different perspectives. Because the eyes' separation is fixed, the brain can rely on systematic differences between the two images (like moving a camera about 6 cm sideways)
Binocular retinal disparity
The relative positions of the two images of an object on the two retinae
Binocular and retinal disparity are used interchangeably.
Ex. the thumb demo, closing one eye and then the other shows the background 'jump' because each eye's image is in a different position.
Where does a fixated object fall on the two retinae?
On the fovea of each eye (the same relative position in both)
Fixating means pointing your eyes so the object's image lands on both foveae. That is the starting reference for working out disparity of everything else.
Horopter
An imaginary circle through the fixation point; objects on it fall on corresponding points in the two retinae
Ex. with your eyes on a box of 3D glasses, a Twizzler on the invisible circle through the box has the same distance from the fovea in both eyes.
Corresponding points and zero disparity
Points whose images fall on the same position in the two retinae relative to the fovea; objects on the horopter have zero disparity
Zero disparity means the image is in the same relative position in both eyes. Anything on the horopter does this.

Images of objects that are NOT on the horopter
They fall on non-corresponding points on the two retinae, giving a disparity different from zero
Anything inside or outside the horopter falls on non-corresponding points, and that nonzero disparity is what carries depth information.

How do the near thumb and the object behind fixation move?
The near thumb and the object behind fixation shift in opposite directions: viewing with the left eye the image spins counterclockwise around fixation, and with the right eye clockwise
Left eye: what is behind fixation is on the left, and what is in front crosses over to the right.
Right eye: the reverse. This is the intuition behind crossed and uncrossed disparity.
Crossed disparity
Disparity for objects nearer than fixation (inside the horopter), where the image position crosses over fixation between the eyes
Closer than fixation = crossed. (The professor admits the names are confusing: crossed = nearer, uncrossed = farther.)

Uncrossed disparity
Disparity for objects farther from you than fixation (outside the horopter)
Farther than fixation = uncrossed disparity, where the image does not have to cross over the fixation point between the eyes.

Magnitude of disparity, how does it relate to distance form the horopter?
How far apart an object's two retinal images are; the farther the object is from the horopter, the greater the disparity
In the thumb demo, the farther the back thumb is from the fixated thumb, the bigger the difference between the eyes' views.
Two properties of disparity
Type (crossed, uncrossed, zero): tells you whether the object is nearer or farther than fixation
Magnitude tells you how far from fixation.
Together they let the brain reconstruct 3D relationships from two 2D images.
Angular disparity
The difference in orientation or angle between two objects, images, or perspectives
D = angle in the right eye − angle in the left eye (the eyeball is a sphere, so position is measured in angles)
D = ∠R − ∠L.

What is the sign of angular disparity for crossed vs. uncrossed disparity?
Crossed < 0; uncrossed > 0
Examples from the slides: uncrossed 10 − (−10) = 20 (> 0); crossed −50 − 60 = −110 (< 0). Zero disparity is 0.
Limits of binocular single vision
Objects very far from the horopter are not fused into a single image; you start to see double
Move one thumb farther from the fixated one and it eventually splits into two. We usually don't notice this in daily life.
Why does the visual system compute precise binocular depth mainly for things around fixation?
Monocular cues handle very near and very far objects well enough, and fixation is what you interact with, consistent with devoting more resources to the fovea
The system doesn't reconstruct the whole 3D scene from disparity. It computes precise depth for what you are looking at and what is near it, and relies on monocular cues for the rest.
Correspondence problem
How the brain figures out which point in one eye's image matches which point in the other eye's image, so disparity can be computed
With cluttered scenes and different images in the two retinae, the brain must work out what matches what (think pixel by pixel) before it can measure disparity.
Hypothesis 1 correspondence
The brain does 2D object recognition in each eye first, labels objects, then compares the objects' positions to get disparity
Under this hypothesis, recognition comes before depth: find the spoon in each retinal image, compare locations, then compute depth.
Hypothesis 2 correspondence
The brain matches parts of the two images using simple properties (color, orientation, spatial frequency) to compute depth, and recognition comes after
Depth comes first and object recognition can follow.
Stimulus for 2 hypotheses
Stimuli containing no objects in either single image, in which an object is only visible if depth (disparity) is computed first
If you can still see depth with no identifiable objects, then object recognition isn't required first. The stimuli have to simulate retinal disparity under tight control of which eye sees which image.
Stereoscope
A device invented by Charles Wheatstone that shows each eye a slightly offset image separately, simulating retinal disparity (the View-Master works the same way)
A divider isolates the two eyes so each sees only its own picture. Two photos taken from slightly different positions produce an experience of depth.
Anaglyph
Two pictures taken about 6 cm apart are given complementary color filters (red and cyan/blue) and overlapped; glasses with matching filters send each image to just one eye
The color-filter glasses act like a stereoscope: the red filter passes one image to one eye and the blue/cyan filter passes the other to the other eye. Some people with astigmatism have trouble seeing the effect.
Random dot stereogram
Two images of random dots that are identical except that a region in one is shifted slightly sideways relative to the other, so no object is visible in either image alone
Julesz (Bell Labs) shifted a square region of dots in one copy and filled in the gap with random dots. Without disparity there is just noise.

Random dot anaglyphs examples
Depth can be computed without recognizing objects, by matching low-level features between the eyes (Hypothesis 2)
With no identifiable object in either eye's image alone, the shape appears only after the brain computes disparity. Depth perception doesn't need objects, and recognition can come after.
Stimulus viewing with the naked eye (no glasses)
A meaningless pattern of dots with no visible shape
You can't tell the two random dot images apart by eye. Only when each eye gets its own image does the brain compute the disparity and a shape appears.
Marr & Poggio's first assumption for solving the correspondence problem
Each feature in one retinal image will match only one feature in the other
It would be unusual for one object to project one image in one eye and several in the other, so the system assumes one-to-one matches.
Marr & Poggio's second assumption for solving the correspondence problem
Visual scenes consist of smooth, continuous surfaces with relatively few abrupt changes in depth
Objects and their parts tend to be smooth, consistent, and close together rather than scattered at wildly different depths. Together with assumption 1, this limits where the brain has to search for a match.
What features does the visual system use to match points between the two eyes
Very simple image properties such as color, orientation, and spatial frequency
Matching on low-level properties, constrained by the two assumptions, narrows the search so the brain doesn't have to look all over the retina.
Binocular neurons
Neurons in V1 (and elsewhere) with receptive field locations in both eyes, which respond best when the preferred stimulus falls on both receptive fields
Neural basis of stereopsis
A neuron's preferred stimulus is the professor's face. It responds a little with the left eye alone, a little with the right eye alone, and a lot with both eyes open while looking at her face. What does this tell you about its receptive fields?
It has receptive fields at the foveae of both eyes (corresponding points), and responds best when the preferred stimulus falls on both
Fixating her face puts it on both foveae, which are corresponding points. The strongest response with both eyes open shows it is a binocular neuron.
A binocular neuron likes the brown cactus. It responds most when the cactus is in front of and slightly to the right of the fixated green cactus (and little when it is on the horopter). What does this tell you?
The neuron's receptive field locations in the two eyes are at non-corresponding points, so it is tuned to a particular disparity (a particular depth relative to fixation)
Only an object at that depth and position projects to both receptive field locations at once. Binocular cells like this are found all over the brain, and their disparity signal also feeds the dorsal pathway.