Senses and Perception: Vision

Week 6-7: Senses and Perception

Lecture 6A: Sensory Systems & 6B: Vision

• How do we measure sensory experiences?
• How do they relate to brain activity?
• The complex circuitry of the eye’s retina.
• From eye to brain: serial and parallel connectivity in the visual pathway.
• Eye movements and visual perception: when and what do we see?

Lecture 7A: Sensory Modalities & Practical 2: Senses and Perception

Lecture 6B: Vision - Prof Natalie Hempel de Ibarra

• Light and the eye
• From retina to cortex
• Eye movements
• Receptive fields in the retina
• Primary visual cortex (V1)
• Cortical streams

Light and the Eye

• Light sources: sun, stars, heated objects, bioluminescence
• Visible light spectrum
• Sunlight is filtered through the atmosphere and reflected from surfaces
• Light is electromagnetic energy that has properties of waves (light ray radiation) as well as charged particles (photons or quanta)
• Differs in intensity and wavelength

Evolution of Eyes

• Eyes come in many forms; advanced types have evolved several times in the animal kingdom.
• Fossil records of image-forming eyes date back to the Cambrian explosion (540 mya).
• Faster movement and navigation in animals required better vision, which selected for eye types with higher spatial resolution.
• More neural resources are required to process a larger amount of information in image-forming eyes.
• Humans and many animals have image-forming eyes.

Visual Field & Retinal Projection

• Visual field: Area of space in which the eye sees its surroundings (visual scenes, objects, or people).
• Retinal projection: A small, inverted 2-D image distorted by the curvature of the eye.
• Perceived image: 3-D, large, upright, stable, non-distorted, colorful.
• Large-sized eyes and brains have the capacity to process more visual information.
• Conscious vision in humans requires an intact retina, thalamus, and primary visual cortex.

Light Intensity & Vision Types

• Intensity of sunlight changes between day and night.
• Photopic vision: 10 candela/m2 to 900,000,000 cd/m2 (bright sun), 300,000 cd/m2 upper limit of vision.
• Mesopic vision: below 10 cd/m2 to 0.0001 cd/m2 (white paper in bright moonlight, 15 min after sunset).
• Scotopic vision: moonless clear night sky (without light pollution), snow/grass in starlight.
• 1 L (lambert) = {1}/{\pi} \text{ candela}

Cones vs. Rods

• Cones have high sensitivity thresholds, therefore respond to bright light (day vision).
• Rods have low sensitivity thresholds, therefore respond to low light (night vision).
• Duplex retina: Instead of extra eyes, many vertebrates have a duplex retina for both day and night vision.

Duplex Retina & Sensory Adaptation

• Circuitry in the duplex retina contains different types of visual interneurons for switching between night and day vision.
• Sensory adaptation: During day or night, when light changes, sensitivity thresholds increase or decrease over time in cones or rods, respectively. This maximizes contrast coding for better vision within the prevailing light range.

Rods, Cones & Opsins

Rods Cone

• Rods and cones differ in morphology, sensitivity thresholds, and wavelength selectivity.
• Three functional classes of cones (S-, M-, and L- cones): Cone opsins differ in their wavelength-specific affinity to absorb light (S, M, and L opsins). Only one opsin type is expressed per cone in the human retina.
• Opsin: A light-sensitive protein (G-protein coupled receptor molecule) in the membrane of photoreceptors which is bound to the chromophore retinal (needed for transduction).
• One functional class of rods: All rods express the same type of opsin (RH1, or rhodopsin).
• Cone circuitry connects to color-coding pathways, whereas rods do not (e.g., we do not see colors at night).

Acuity & Fovea

• Central fovea is active during the day and in bright light.
• Acuity (ability to resolve spatial details) is proportional to the density of receptor cells.
• Acuity of vision is highest in the fovea and decreases towards the periphery of the retina.
• Central fovea only contains cones.
• At night, high acuity is sacrificed for sensitivity, and it is more advantageous to have no rods in the fovea.

Analogies Between Evolution and Engineering of Visual Sensors

• Lens to focus image
• Aperture to control light entering (Iris)
• Pixels to register image (photoreceptors)
• Filtering media (glass body, macula, pigment)
• Filter to protect lens (cornea)
• Lens cover for when not in use (eye lid)
• Cleaning mechanism (tears)
• Processing algorithms (retinal interneurons)
• Modern cameras are still a poor technical imitation of the retina, which has a much larger sensor area and much more sophisticated processing circuits.
• The first steps in processing an image: pixels and filtering algorithms.

Impressionism, Art, Science & Psychology

• Europe, late 19th century: Impressionist art movement
• Impressionism: Artists captured perceptual qualities of light, color, and atmosphere reflecting. New color technologies, materials, and dyes becoming available.
• At the same time, scientists discover principles of physics and perception of light. Psychology emerges as a scientific discipline (e.g., psychophysics, experimental psychology), studying visual perception and consciousness, but also emotion and memory.
• Art and science influence each other’s explorations and developments.
• Emile Zola (1880) referred to impressionist artists as those who "propose to leave the studio, where painters have been immured for centuries, to go and paint out-of-doors. .. This study of light, in its thousands of decompositions and recompositions, is what has been called more or less appropriately impressionism, because a painting becomes as a result the impression of a moment felt before nature."
• Cataract (clouding of lens in the eye) – frequent but well-treatable disease caused mainly by aging (onset at age of 30-40 years), but also diet and UV exposure. Contributes up to 5% of blindness in countries of Global North but 60% in Global South due to lack of access to medical care.

Why Do We See What We See?

• We sometimes notice discrepancies between reality and perception; it is not a failure of our senses or brain.
• The sensory systems and brain resolve ambiguities in the sensory environment.
• The brain saves energy and numbers of neurons by remembering and predicting, depending on the environment, context, and task.

Serial Connections of Visual Pathway in the Retina

• First steps of processing in the retina of the eye:
  1. Photoreceptors to bipolar cells (signals transmitted as graded potentials).
  2. Bipolar cell to ganglion cells.
  3. The long axons of the ganglion cells form the optic nerve that leaves the eye and transmits action potentials to the thalamus and other brain areas.
• Cross-connections between in retinal layers:
  • Horizontal cells (inputs from photoreceptors and projections to bipolar cells).
  • Amacrine cells (inputs from bipolar cells and project to ganglion cells).
• Human retina contains approximately 100 million rods and 4 million cones and only 1 million ganglion cells (convergence).

Visual Pathways

• Geniculate-striate visual pathway (required for conscious vision in humans):
  Retina – LGN (lateral geniculate nucleus) of the thalamus – V1 (primary visual cortex) – areas of the higher visual cortex (90% of retinal projections), V1 required for conscious visual experiences.
• Extrageniculate pathways, for example:
  Retina – superior colliculus (SC) – pulvinar nucleus of the thalamus (pulvinar) for eye movement control and visual attention (10% of retinal projections).

Retinotopic Projections from Retinal Ganglion Cells

• Retinal ganglion cells project retinotopically to each layer of the LGN (an example of neural mapping).
• Right and left eye projections are also segregated in the LGN.

Eye Movements: Saccades and Fixations

• Saccades (jumps) and fixations (stops) direct fovea to collect information about the visual scene.
• The field of view is defined by the position and orientation of the eyeball, of the head, and of the body.
• 2-3 saccades per second.
• Task influences eye movement patterns.

Control of Eye Movements

• Automatic control of eye movements comes from the superior colliculus (SC).
• Conscious control of eye movements comes from the cortical frontal eye fields (FEF).

Eye Movements in Everyday Behavior

• Saccades: Move the eye very quickly to a new position between periods of gaze stabilization (fixations) in order to scan the scene across the entire field of view.

Atypical Eye Movements in Dyslexia

• Difficulties in reading words, sentences, text.
• Longer durations of fixations and shorter saccades, more fixations during reading.
• Shorter visual attention span impacts on eye movement patterns.

Spatial Relationships and Properties of Objects

• Without context cues, we perceive the physical reflectance of the surfaces which carries little information.
• Edges and shadows provide context information about the spatial structure of objects (in the picture a three-dimensional object with inclined surfaces) or spatial relationships between objects (identical objects laying sidewise behind the central one).

Smart Computations in the Retina

• Functional classes of cells in the retina:
  • 4 types of photoreceptors (3 cone types and the rods).
  • 50-70 ctypes of horizontal, bipolar, and amacrine cells.
  • 20-30 types of ganglion cells.
• First stages of visual processing with inhibitory (•) and excitatory (•) synapses in neural circuits of retina:
  • Edge detection in visual scenes.
  • Edge enhancement in patterns.
  • Filtering of spatial, wavelength, movement, and directional information.

Mach Bands

• Mach bands (optical illusion described by physicist Ernst Mach in 1865)
• Stripes with low-contrast edges

Receptive Fields of Bipolar and Ganglion Cells

• Convergence:
  • Fovea: 1 cone to 1 bipolar cell.
  • Periphery of the retina: Many cones to 1 bipolar, many bipolars to 1 ganglion cell.
• Acuity is high in fovea (low convergence) and low in the periphery of the visual field (high convergence).
• Cones that converge on a bipolar cell form the bipolar cell’s receptive field. Similarly, the receptive field of a ganglion cells is formed by all converging bipolar cells.
• Many types of receptive fields (e.g., simple, center-surround).

ON/OFF Cells

• Signals of ON-/OFF- cells change with ratio of light/dark
• Whilst the ON-center bipolar cell depolarizes, the ON-center ganglion cell responds by increasing its spike rate.
• Whilst the OFF-center bipolar cell hyperpolarizes, the OFF-center ganglion cell responds by decreasing its spike rate.
• Intermediate graded potential (bipolar cell) or spike frequency (ganglion cell) when illuminated with a uniform light stimulus.

Filter Mechanisms in the Retina

• Why two types of center-surround receptive fields?
  • Objects can be dark against a bright background, or bright against a dark background.
  • Bipolar and ganglion cells with ON-center/OFF-surround receptive field.
  • Bipolar and ganglion cells with OFF-center/ON-surround receptive field.
  • Are neural circuits that combine excitatory and inhibitory synapses.

Columnar Structure of Primary Visual Cortex (V1)

• Hypercolumn (appx 1mm2) is composed by:
  1. One left-eye and one right-eye ocular dominance column (L, R).
  2. Several orientation columns (rainbow colors) containing simple and complex cells that respond to orientation of shapes, such as a bar.
  3. Blobs (drawn as cylinders) are structures in layers II-III of the V1 and are involved in color vision.
• Retinotopic organization: The spatial mapping arising from the projection of the image onto the retina is preserved also in the V1.
• In addition to the six horizontal layers, the neurons in V1 are further segregated into functionally distinct hypercolumns.

Responses of Neurons in the Orientation Columns of V1

• When recording from neurons of a particular orientation column in V1, these neurons respond to the orientation of a bar stimulus only within a small part of their receptive field (which corresponds to a small part of the visual field).
• Different from the retinal ganglion cells, these neurons fire at the maximal spike rate when a bar stimulus shows their preferred orientation.
• Other cortical cells in V1 respond with the maximal spike rate to a preferred direction of motion of bars or patterns.

Functions of Simple and Complex Cells

• Analysis of contours and boundaries analysis of objects
• Shape and positional invariance
• Contour enhancement for object identification
• \implies V1 is fundamentally important for conscious vision and perception

Unconscious Vision in Blind Humans and Blind Primates

• Damage to V1 causes cortical blindness, the loss of conscious vision. Patients are able to perform visually-guided behaviors, like grasping or pointing to the location of objects, or avoiding obstacles, correctly at a level above chance. This is known as blindsight.

Visual Areas in the Macaque Cortex

• Higher visual areas in the cortex
• V1, V2, V3, V4, MT (V5), etc.

V4 Neurons

• V4 neurons respond to more complex stimuli compared to V1 and V2.

Object Recognition

• Discrimination (<200 ms)
• Recognition of objects (also when changes in object position, size, viewpoint, and visual context)
• Categorization

Ventral Cortical Stream

• Critical for object recognition

Two Visual Streams in the Cortex

• Dorsal stream (where system): Interacting with the world (via V5/MT)
• Ventral stream (what system): Making sense of the world (via V4)

Seeing for Action: Eye-Hand Coordination

• Guiding hand movements requires two processes:
  • Deciding which objects to interact with
  • Interacting with objects skillfully
• These processes require different types of information from both the dorsal and the ventral streams.

Parallel Visual Streams and Cross-Connections

• Thick arrows: main connections
• Thin arrows: many connections are reciprocal
• Pathways that bypass V1 have been accounted to mediate blindsight and residual capabilities for visual discrimination in blind people and monkeys.

Subconscious Vision and Emotions

• SC-mediated pathways are interconnected with amygdala
• Rapid processing of emotional information, in particular for salient stimuli, such as faces and snakes
• Short-cut to drive motor actions, such as fast orienting eye movements

Key Points

• Retina: first stages of visual processing (edge detection in visual scenes, edge enhancement in patterns, filtering of spatial, wavelength, movement, and directional information).
• Lateral inhibition in retinal cells is responsible for edge enhancement (and the Matchband effect).
• Edges and shadows provide context information about the spatial structure of objects or the spatial relationship between objects.
• Cones/rods converging on a bipolar cell form its receptive field, while cones/rods and bipolar cells converging to a ganglion cell form the ganglion cell’s receptive field. Receptive fields are larger in the periphery (blurrier vision due to lower acuity) and smaller in the fovea (helps to achieve highest acuity).
• Some classes of bipolar and ganglion cells have a center-surround receptive field. These can be ON-center/OFF surround (lateral inhibition from receptors in the surround) or OFF-center/ON-surround (lateral inhibition from the photoreceptors in the center of the center-surround receptive field).
• Ganglion cells respond to ratios of light/dark (e.g., a small dot of light), but not to uniform illumination.
• P and M ganglion cells project to different layers in the LGN (and V1) and have different properties (receptive field sizes, conduction speed, acuity, presence/lack of color sensitivity).
• P and M ganglion cells project retinotopically to segregated layers in the LGN.
• V1 has a columnar structure, with neurons mapped and segregated in hypercolumns which combine orientation columns and ocular dominance columns for each part of the visual field. All neurons in an orientation column share the same preference for a particular orientation of a bar stimulus in their receptive field. Within a hypercolumns orientation columns are found together if they receive input from either the left or right eye, thereby forming a pair of left-eye and right-eye ocular dominance columns in each hypercolumn.
• Simple cortical cells (aka bar detectors or edge detectors): respond best to an edge or a bar of particular width, orientation, and location in the visual field. Complex cortical cells: respond best to a bad or particular size and orientation anywhere within a particular area of the visual field.