Notes on Visual Processing: D.F., Pathways, V1, and Functional Modules
Case Study: D.F. and the Visual Processing Dissociation
1988: A young woman known as D.F. fell into a coma due to carbon monoxide poisoning from a faulty propane heater.
After waking, she appeared alert but could not see: diagnosed with cortical blindness (damage to primary visual cortex, back of the brain).
Rapidly, some aspects of vision returned:
Could see colors (e.g., red and green of flowers).
Could see textures and fine details.
Could perceive motion.
But she could not interpret visual information as recognizable objects: could not see shapes or identify objects by sight, even after years.
D.F.’s object recognition impairment contrasted with normal ability to identify objects by touch, indicating a visual impairment, not a cognitive one.
Key finding: some brain areas support object recognition while other areas support action guided by vision; lesion affected object-recognition areas but spared vision-for-action areas.
Implication for brain organization: supports functional specialization and separate pathways for recognizing objects vs guiding actions.
Early in the chapter, this case motivates the distinction between ventral stream (what/recognition) and dorsal stream (where/how to act), later refined with D.F.’s case to emphasize ventral pathway damage affecting perception but not action.
Core Concepts in Visual Processing
Two overarching principles organize the brain’s visual pathways:
Functional specialization: different neural pathways/areas are specialized for different kinds of visual information (shape, color, motion, depth, etc.).
Retinotopic mapping: neighboring retinal locations map to neighboring neurons in each visual area, preserving spatial layout of the scene.
Early visual processing components:
The retina transduces light into neural signals via photoreceptors.
Lateral inhibition in retinal circuits enhances edges at locations with abrupt changes in intensity.
Retinal ganglion cells convey elementary visual features (color, edge orientation) via the axons that form the optic nerve.
Higher-order processing requires more neural processing to create representations of shapes, locations, and identities of objects in a scene.
Progression of signals: retina → subcortical structures (LGN, SC) → primary visual cortex (V1) → higher visual areas (V2, V4, IT, MT, etc.).
Ventral pathway: forms a pathway for recognizing objects (the “what” pathway).
Dorsal pathway: forms a pathway for visually guided action and spatial localization (the “where/how” pathway).
Blindsight: the superior colliculus (SC) can support some visually guided actions even when V1 is damaged, through alternative pathways.
From Eye to Brain: The Visual Pathways
Visual field information is mapped contralaterally due to optic nerve crossing at the optic chiasm:
Left visual field → right hemisphere; Right visual field → left hemisphere.
In each hemisphere, the majority of optic tract axons go to the LGN (> ~90%), with others going to the superior colliculus and other targets.
The optic nerve carries signals from retinal ganglion cells (RGCs) to the brain; there are about 1{,}000{,}000 axons from the two eyes combined in humans, forming the optic nerves.
The contralateral representation of visual space is a fundamental organizational principle (opposite-side organization vs. ipsilateral organization).
After LGN, signals are relayed to the primary visual cortex (V1). Some signals branch to the superior colliculus (SC) for rapid eye movements and multisensory integration.
The Lateral Geniculate Nucleus (LGN)
LGN is part of the thalamus; there is one LGN in each hemisphere.
Structure: six layered organization with distinct cell sizes and properties:
Magnocellular layers (layers 1–2): large cell bodies; projection from parasol RGCs; specialized for dynamic visual properties (motion and flicker).
Parvocellular layers (layers 3–6): small cell bodies; projection from midget RGCs; specialized for static properties (color, texture, form, depth).
Koniocellular layers: interleaved between magnocellular and parvocellular layers; inputs from bistratified RGCs; also involved in color processing.
Retinotopy: each LGN layer forms a retinotopic map; maps align so that adjacent retinal locations map to adjacent LGN neurons, and maps across layers line up.
Input specificity: each LGN layer receives signals from one eye only; contralateral/ipsilateral eye input follows a layer-specific pattern: layers 1, 4, 6 (and koniocellular under them) receive from the contralateral eye; layers 2, 3, 5 (and their koniocellular layers) receive from the ipsilateral eye.
Functional specialization evidence:
Magnocellular layers: sensitive to motion and flicker; less sensitive to color details.
Parvocellular layers: sensitive to color, texture, shape, depth; less sensitive to motion.
Koniocellular layers: carry color information; role in color processing.
Attention and LGN: attention can modulate LGN activity; fMRI studies show greater LGN activity when attention is directed to a visual stimulus (e.g., a checkerboard) vs. a central task, suggesting top-down feedback from higher cortical areas can influence LGN processing.
Overall role of LGN: serves as a gateway and a modifiable relay for transmitting visual information from the eyes to the cortex; is subject to top-down attentional control, effectively acting as a “volume control” for incoming information.
The Superior Colliculus (SC)
Location: a small structure near the top of the brain stem (one in each hemisphere).
Primary role: rapid control of eye movements toward visual targets; integrates visual information with auditory and somatosensory signals.
Neuronal properties: SC neurons respond to many visual features but are especially tuned to location rather than object identity; they help shift gaze quickly to salient locations.
Multisensory integration: SC neurons can respond to combined visual-auditory or visual-tactile stimuli, often showing stronger responses to multisensory combinations than to single-modality stimuli.
Pathways: SC sends signals to the visual cortex beyond V1 and participates in pathways that bypass V1, relevant to blindsight phenomena.
Primary Visual Cortex (Area V1)
V1 receives signals from the LGN (mainly from layers 4Cα and 4Cβ, with koniocellular inputs to 2/3) and is organized into columns and layers.
Two main cell types in V1:
Simple cells: respond best to a bar of light at a particular orientation and location; receptive fields are elongated; their response depends on the bar aligning with excitatory centers of inputs from LGN/retina.
Complex cells: respond to oriented bars but with broader locations; less dependent on exact retinal location; sensitive to a range of positions within their receptive fields.
Orientation tuning: simple cells have orientation preferences; orientation tuning curves show peak responses at their preferred orientations; population coding (thousands of cells with different preferred orientations) allows robust orientation decoding despite contrast differences.
Luminance contrast and orientation: a single simple cell’s response can be ambiguous due to varying bar contrast; population coding helps disambiguate orientation from contrast by considering responses of many cells with different orientation preferences.
End-stopped cells: respond more strongly to longer bars up to a certain length, then reduce response; important for detecting object corners and ends.
Color and motion processing in V1: many V1 neurons are color-tuned; ~30% of V1 neurons are direction- and speed-tuned; many neurons are tuned to edge length and depth; binocular (stereoscopic) cells respond to disparity between the two eyes.
Population coding extends to multiple features: neurons in V1 can be jointly tuned to orientation, color, motion, binocular disparity, etc.; the brain uses patterns of activity across populations to infer multiple stimulus properties.
Receptive field organization: V1 is organized in a retinotopic map; small cortical areas correspond to small regions of the retina; foveal representation is magnified, reflecting cortical magnification.
Organization of V1 and Related Cortical Columns
Cortical columns: vertical organization through layers that group neurons with similar response properties; key types include:
Ocular dominance columns: neurons within a column preferentially respond to input from one eye; alternating columns respond to left vs right eye input.
Orientation columns: neurons within a column have similar orientation tuning; as one moves laterally through adjacent columns, preferred orientation changes systematically (often forming a pinwheel pattern).
Retinotopic mapping: adjacent cortical locations correspond to adjacent retinal locations; columns at neighboring locations map to neighboring retinal regions.
Ocular dominance columns visualization: tracer studies show alternating stripes in V1 corresponding to input from each eye; border zones contain neurons with binocular disparity tuning.
Orientation columns: systematic progression of preferred orientations across cortex; electrode mapping reveals gradual shifts in preferred orientation as the electrode traverses through adjacent columns.
Retinotopic maps and cortical magnification: V1 displays a retinotopic map with disproportionately large cortical representation for the fovea (cortical magnification); this is visible in retinotopic mapping studies using fMRI and other imaging methods.
Blob and interblob organization: V1/ V2 organization includes blobs (color processing areas in layer 2/3) and interblob regions; V2 contains thin and thick bands separated by pale bands; these structures reflect functional specialization beyond V1.
Functional Areas, Pathways, and Modules
Four major functional modules along the ventral and dorsal streams:
Ventral pathway (the what pathway): V1 → V2 → V4 → Inferotemporal cortex (IT) → specialized areas such as fusiform face area (FFA) and parahippocampal place area (PPA)
Dorsal pathway (the where/how pathway): V1 → V2 → MT (V5) → parietal cortex; includes areas within the intraparietal sulcus (IPS) like LIP, AIP, MIP
V4: color and curvature processing; intermediate stage for form representation; neuronal tuning to color and edge curvature; lesions/deficits can lead to achromatopsia (color blindness) when color processing is disrupted.
IT cortex: object-selective region; neurons respond to complex shapes and objects; some IT neurons respond to faces (face-selective areas like FFA); larger receptive fields often cover substantially large visual areas.
Lateral Occipital Cortex (LOC) and IT: object recognition areas; respond to shapes and object categories; FFA and PPA are subdivisions within IT cortex.
FFA (Fusiform Face Area): selectively responds to faces; lesions impair face recognition (prosopagnosia).
PPA (Parahippocampal Place Area): selectively responds to large-scale places (landscapes, buildings, rooms).
MT (Area MT / V5): motion processing module; neurons tuned for direction and speed of motion; fMRI shows MT activation with moving dot stimuli; MT damage impairs motion perception.
Intraparietal sulcus (IPS): visually guided actions; divisions include:
LIP (Lateral IPS): eye movements and attention shifts
AIP (Anterior IPS): grasping; polymodal integration for hand shaping
MIP (Medial IPS): reaching movements; motor planning for reaching
The dorsal/ventral division (Ungerleider & Mishkin, 1982) originally posited two streams with specialized lesions causing task-specific deficits: dorsal for where/how and ventral for what; later refined with D.F. patient studies and Goodale & Milner’s perspective emphasizing perception-action distinctions.
The interplay of perception and action: perception can be dissociated from action; ventral pathway supports conscious perception of shape/identity, while dorsal pathway supports guiding actions even when perceptual awareness is impaired.
Feedback and interactions: actual brain organization includes extensive feedback between areas; pathways are not strictly segregated but are interacting networks.
Case Study: Goodale & Milner and D.F. (Ventral vs Dorsal Pathways)
Ungerleider & Mishkin proposed dorsal (where) vs ventral (what) pathways based on lesion studies in monkeys.
D.F. case (Goodale et al., 1991): ventral pathway damage due to carbon monoxide poisoning; she suffered visual agnosia (inability to consciously recognize objects by sight) but could guide actions towards objects (e.g., grasping) with high accuracy.
Two tests:
Posting task: D.F. could insert a card into a rotatable slot with near-control-like accuracy, indicating intact action-oriented processing for orientation information.
Perceptual matching task: D.F. performed near chance level in matching the orientation of the slot, indicating a perceptual deficit.
D.F.’s grasping accuracy relied on ventral-to-dorsal information flow, showing dorsal pathway access to early visual information for action even when conscious perception was impaired.
Optic ataxia (parietal lobe lesions): deficits in visually guided actions while basic object recognition remains intact, illustrating the complementary pattern to D.F.
Milner & Goodale’s interpretation: dorsal pathway is the “how” pathway, coordinating perception with action; both pathways are essential and interact, not strictly independent.
Population Coding and Feature Representation in V1
V1 encodes multiple visual features beyond orientation, including color, motion, length, size, and depth.
End-stopped cells and binocular disparity contribute to depth perception.
Population coding: a single neuron cannot unambiguously determine a stimulus feature when other features vary (e.g., orientation vs. contrast). A population of neurons with different orientation preferences resolves this ambiguity.
Example: Two simple cells with different preferred orientations respond differently to bars of light at various orientations and contrasts. Across many cells, consistent response patterns decode orientation independent of contrast.
Simple Cells A (preferred ~90°) and B (preferred ~75°) show patterns where A > B for a set of orientations; this relative pattern remains consistent across contrast levels, allowing orientation to be inferred from population activity.
When orientation changes, the relative responses swap accordingly, illustrating that orientation information is encoded by the pattern across a population of cells.
The same population coding principle applies to multiple features simultaneously, enabling the brain to parse orientation, color, motion, and depth through complex, distributed activity.
Major Pathways: Ventral vs Dorsal
Ventral Pathway (the what pathway):
V1 → V2 → V4 → IT → specialized object representations (including LOC, FFA, PPA)
Key function: form and color processing for object recognition; supports conscious perception of identity and category.
Dorsal Pathway (the where/how pathway):
V1 → V2 → MT (V5) → parietal cortex (IPS: LIP, AIP, MIP)
Key function: motion processing, spatial localization, and visually guided actions; coordinates perception with action (e.g., reaching, grasping, eye movements).
The pathways are supported by parallel anatomy:
Midget RGCs → Parvocellular LGN → V1 blobs/interblobs → V2 thin/pale bands → IT (color/form processing).
Parasol RGCs → Magnocellular LGN → V1 4Cα → V2 thick bands → MT (motion processing).
Bistratified RGCs → Koniocellular LGN → V1 blobs → V2 thin bands → color processing.
The two pathways are not completely segregated; there is inter-area communication and feedback that modulates processing and perception.
Visual Prosthetics and Real-World Applications
Many people are legally blind due to retinal or optic diseases; damage to cortex is less common and brain-based prosthetics aim to bypass damaged retina/eye.
Visual neuroprosthetics involve: capturing visual information with a camera, processing signals, and stimulating the visual system directly (retina, optic nerve, or cortex) with implanted stimulators.
Cortical stimulation of V1 can evoke small, star-like percepts whose perceived size depends on retinotopic location in V1 due to cortical magnification.
Challenges for prosthetics:
Achieving high spatial resolution: the human optic nerve has over 1{,}000{,}000 RGC axons, while cortical implants may use only a few hundred electrodes, yielding coarse resolution.
Experimental implementations include 64-electrode modules or 152-electrode arrays in V1 in monkeys; spatial resolution remains far below natural vision.
A proposed approach uses a proportional square array of electrodes in V1, stimulating foveal regions with fewer electrodes and peripheral regions with more electrodes to reflect cortical magnification.
Demonstrations include simulating perceivable text such as the words "FIAT LUX" by mapping image sections to electrode activations.
Outlook: early devices may enable basic light/direction detection or motion perception; higher-level functions such as shape and pattern recognition may become feasible with technological advances within the next decade(s).
Key Equations and Quantitative References
Contralateral representation of visual space (schematic):
Left visual field → right hemisphere; Right visual field → left hemisphere.
LGN input proportions:
Approximately 90 ext{%} of optic tract axons project to the LGN; the remainder go to the SC and other targets.
Population coding for orientation (illustrative):
Let Ri be the response of simple cell i with preferred orientation θi; a population code can estimate the stimulus orientation θ̂ via
hetâ = ext{atan2}igg( rac{ extsumi Ri \, ext{sin}(2 hetai)}{ extsumi Ri \, ext{cos}(2 hetai)}igg)
This encodes orientation despite variations in contrast or other features across cells.
Cortical magnification: foveal representations are disproportionately represented in V1; the density of V1 neurons is higher for the fovea and receptive fields are smaller, yielding higher acuity for central vision.
Review Questions and CHECK YOUR UNDERSTANDING
3.1: Contralateral representation of visual space – draw a diagram showing an object in the right visual field, identify retinal locations in both eyes, and trace pathways to the primary visual cortex.
3.2: Summarize the pattern of connections from retinal ganglion cells to LGN layers (magnocellular, parvocellular, koniocellular) and then to V1.
3.3: Summarize functional differences among magnocellular, parvocellular, and koniocellular LGN layers.
3.4: State the main function of the superior colliculus and why it’s considered a multisensory integration site.
3.5–3.9: Answer questions about simple vs. complex cells, orientation tuning, luminance contrast effects, population coding, end-stopped cells, binocular disparity, and how the brain disambiguates multiple features via population codes.
3.10–3.13: Define cortical column, ocular dominance column, orientation column, retinotopic mapping, and cortical magnification.
3.14–3.16: List four characteristics to compare visual areas, contrast dorsal vs ventral pathway information, and interpret DF’s dissociation between perception and action.
3.17: Match stimuli (faces, simple/complex shapes, visually guided action, color/edge curvature, houses/landscapes, motion) to areas (V4, MT, IT, FFA, PPA, IPS).
Applications and Summary
The visual system is organized as a network of specialized, interacting modules arranged along two major pathways, each serving distinct functional roles:
Ventral (what): form and color analysis for object recognition; IT/LOC; FFA; PPA.
Dorsal (where/how): motion analysis, spatial localization, and action guidance; MT; LIP/AIP/MIP within the IPS.
The brain achieves robust perception and action through population coding across many neurons with varied selectivities, enabling disambiguation of multiple stimulus features such as orientation, color, and motion.
Real-world implications include understanding visual deficits (e.g., DF’s ventral stream damage) and developing brain-based prosthetics to restore basic visual functions when the eyes or retina are impaired.