Vision

Light sources

• Sun, stars, heated objects, bioluminescence

  • Sunlight is filtered through the atmosphere & reflected from surfaces 

• Light is electromagnetic energy that has properties of waves (light ray radiations) as well as charged particles 

• Differences In intensity 

• Differences in wavelength

Dim lights

rods

bright lights

cones

Human and many animals have image-forming eyes

• Eye comes in many forms

  • simplest → cluster of light-sensitive cells

• many eyes developed to have optic structure (lens & cornea)

• advances types of eye have evolved several times

  • fossil records of image-forming eyes back to Cambrian explosion

• faster movement & navigation required better vision

  • selected morphologies with higher spatial resolution

    • more neural resources are required to process larger information

Larger bodies can sustain larger-sized eyes as brains are capable of processing larger amounts of information

• conscious visual sensations require intact retina, thalamus & primary visual cortex

• visual field

• retinal projection

visual field

• area of space in which the eye sees visual scenes

  • objects people & surrounding background

Retinal projection

• inverted 2d image that is distorted by the curvature of eye

  • after processing the perceived image is 3D large & upright

why do we see what we do

• discrepancies between reality & perception occur

• sensory system & brain resolve ambiguities in sensory environment

  • brain saves energy & neurons by remembering & predicting depending on context/task

• aspects of visual processing in retina are fixed to ensure brain has priors to make decisions

eye seeing/active both day & night

• have two morphological types of visual receptors

  • duplex retina in vertebrate eye

  • cones are specialised for vision during day

  • rods are specialised for vision during night

    • allows us to cope with changing light levels

      • rods are more sensitive

• Ospin

• light-sensitive proteins (G-protein couples molecule receptor)

• in the membrane of photoreceptors bound to the chromophore retina (need for transduction)

  • specifically in the photoreceptor cells called rods and cones.

• play a crucial role in phototransduction

  • the conversion of light into electrical signals that the brain can interpret as vision.

three functional classes of cones

• S-, M-, & L wavelength cones opsins

  • differ in their wavelength-specific affinity to absorb light

    • only 1 opsin expressed per cone

one function class of rods

• all rods expressed same type of opsin (rhodopsin)

processing an image (pixels & filtering algorithms)

1. Lens to focus image 

2. Aperture to control light entering (Iris)

3. Pixels to register image (photoreceptors)

4. Filtering media (glass body, macula, pigment)

5. Filter to protect lens (cornea) 

6. Lens cover for when not in use (eyelid)

7. Cleaning mechanism (tears)

8. Processing algorithms (retinal interneurons)

   • A 108 MP (megapixel) camera is still a poor technical imitation of the retina which has a much larger sensor area and much more sophisticated processing circuits

Dim-light vision (rods) doe not use central fovea

• Acuity (ability to resolve spatial details) is proportional to the density of receptor cells

• Acuity of vision is highest in fovea & decreased towards periphery of the retina

• Eye movements positions the fovea in those position of the visual field where it is most important to collect the most fine-grained visual information

• At night, high acuity is sacrificed for sensitivity 

  • More advantageous to have no rods in fovea

First steps of processing in the retina of the eye 

• hardwired complex → allows brain to make sense of the work & extract edges

• serial connections (start of labelled line in visual pathway)

  1. Photoreceptors → bipolar cells (signals transmitted as graded potentials)

  2. Bipolar cell → ganglion cells

  3. long axons of the ganglion cells form the optic nerve that leaves the eye → transmits action potentials to the thalamus & other brain areas

ganglion cells

• Retinal ganglion cells (RGC) project their axons along the optic nerve

• are responsible for the propagation of visual stimuli to the brain

bipolar cells

• one of the main retinal interneurons

• provide the main pathways from photoreceptors to ganglion cells

  • shortest/direct pathways between the input & output of visual signals in the retina

Horizontal cross-connections

• Horizontal cells receive inputs from photoreceptors & project to bipolar cells

• Amacrine cells receive inputs from bipolar cells & project to ganglion cells

Human retina contains 

• 100 million rods

• 4 million cones

• 1 million ganglion cells

retina to cortex

• Geniculate-striate visual pathway

  • retina → set of axonal connections that project from the lateral geniculate nucleus (LGN) of the thalamus to the primary visual cortex

    • required for conscious vision in humans

    •  areas of the higher visual cortex (90% of retinal projections)

• Extrageniculate pathways

  • Retina → superior colliculus (SC) pulvinar nucleus of the thalamus (pulvinar) for eye movement control & visual attention (10% of retinal projections)

Spatial layout of retinal ganglion cell projections is preserved

• retinal ganglion cells project retinotopically to each layer of LGN

• right & left eye projections are segregated in LGN

retinotopically

• neurons with receptive fields close together in visual space have cell bodies close together in the cortex

Unconscious visions in blind humans & blind primates

• Damage to V1 causes cortical blindness → loss of conscious vision

• Patients are able to perform visually-guided behaviours, like grasping or pointing to the location of objects, or avoiding obstacles, correctly at a level above chance. 

  • known as blindsight

    • Still able to use information 

why do we move our eyes?

• movement called Saccades (jumps) & fixations (stops)

  • 2-3 saccade per second

• direct fovea to collect information about visuals scene

• field of view is defined by position/orientation of eyeball & head/body

• tasks influence patterns

  • Automatic control of eye movements comes from the superior colliculus (SC) 

  • Conscious control of eye movements comes from the cortical frontal eye fields (FEF)

Yarbus (1914-1986) 

• developed the first methods to accurately measure eye movements & viewing behaviour

eye movement in everyday behaviour

• Saccades

  • move the eye very quickly to a new position between periods of gaze stabilisation (fixations) in order to scan the scene across the entire field of view

• Brain has to work out whats important

• When reading can skip word to word 

Atypical eye movement in dyslexia

Prado et al (2007)

• Difficulties in reading words, sentences, text 

• Longer durations of fixations & shorter saccades

  • more fixations during reading 

• Shorter visual attention span impacts on eye movement patterns

receptive fields

• refer to the specific regions of visual space that influence the activity of retinal ganglion cell or a cell in the lateral geniculate nucleus (LGN) of the thalamus

  • determine how the visual system processes visual information from the external world

• organized in a spatially overlapping & tiled manner

  • neighbouring neurons have receptive fields that partially overlap with each other

  • allows for the comprehensive coverage of visual space and the efficient processing of visual information.

2 main types of receptive fields in the retina:

• Center-Surround Receptive Fields

• ON/OFF Receptive Fields

Center-Surround Receptive Fields

• consist of a central region surrounded by a surrounding region → have opposite effects on the neuron's activity.

• "ON-center, OFF-surround" receptive field arrangement → allows neuron to detect contrasts in light intensity

ON/OFF Receptive Fields

• consist of separate regions that respond either to increases (ON response) or decreases (OFF response) in light intensity

• Neurons with ON/OFF receptive fields have distinct subregions that respond selectively to light increments and decrements, respectively.

• enables the neuron to detect both light & dark features within its receptive field.

Identifying spatial relationships & properties of objects

• Without context cues, we perceive the physical reflectance of the surfaces which carries little information

• Edges & shadows provide context information about the spatial structure of objects or spatial relationships between objects

  • (identical objects laying sideways behind the central one)

• Background can affect the colour of an object 

Complex structure of vertebrate retina

• Functional classes of cells in the retina:

  • 4 classes of photoreceptors (3 cone types and rods)

  • 50-70 classes of horizontal, bipolar and amacrine cells 

  • 20-30 classes of ganglion cells

• First stages of visual processing with inhibitory and excitatory synapses in neural circuits of retina

  • Edge detection in visual scenes 

  • Edge enhancement in patterns 

  • Filtering of spatial, wavelength, movement and directional information

Acuity & retinal receptive fields

• Convergence in 

  • fovea: 1 cone → 1 bipolar

  • periphery of the retina: Many cones to 1 bipolar, many bipolar → 1 ganglion cell 

• Acuity

  • high in fovea (low convergence)

  • low in periphery of the visual field (high convergence)

• Cones that converge on a bipolar cell form the bipolar cell’s receptive field

• receptive field of a ganglion cells is formed by all converging bipolar cells

• Many types of receptive fields (e.g. simple, centre-surround)

Filter mechanisms in the retina 

• Filter mechanisms in the retina are neural circuits that combine excitatory and inhibitory synapses

  • Bipolar & ganglion cells with ON-centre/ OFF-surround receptive field

  • Bipolar & ganglion cells with OFF-centre/ ON-surround receptive field

Why two types of centre-surround receptive fields?

• Objects can be dark against a bright background, or bright against a dark background

ON- and OFF-centre cells respond to ratios of light/dark

• Whilst ON-centre bipolar cell depolarises

  • ON-centre ganglion cell responds by increasing its spike rate

• Whilst OFF-centre bipolar cell hyperpolarizes

  • OFF centre ganglion cell responds by decreasing its spike rate

• When at rest, a ganglion cell is not silent but fires action potentials (spikes) at a spontaneous rate

ganglion cells do not respond to uniform illumination

• job to see if there is an edge not how light 

  • when light spot covers ON-centre → ganglion cell responds with highest spike rate

  • When a surrounding light covers all but not the ON-centre → ganglion cell responds with the lowest spike rate/no spikes

  • When the whole receptive field is equally stimulated → ganglion cell fires with an average frequency independently of the light intensity

• in off centre ganglion cell → respond to dots & rings

primary visual cortex (V1)

• columnar structure, 6 horizontal layers with neurons segregated into functionally distinct hypercolumns

• Hypercolumn is composed by

  • 1 left & 1 right eye dominance column

  • several orientation columns w/simple/complex cells which respond to orientation of shapes

  • Blobs in layers ||-||| of V1 & involved in colour vision

Retinotopic organisation

• The spatial mapping arising from the projection of the image onto the retina is preserved also in the V1

Responses of neurons in the orientation columns of V1

• when recording from neurons of an orientation column in V1

  • neurons respond to the orientation of a bar stimulus only within a small part of their receptive field

    • which corresponds to a small part of visual field

• these neurons fire at the maximal spike rate when the bar stimulus shows their preferred orientation

• other cortical cells in V1 respond with maximal spike rate to preferred direction of motion of bars/patterns

functions of simple & complex cells

• Simple cells respond to oriented edges or lines within their receptive fields

• complex cells respond to oriented edges or lines regardless of their position within the receptive field.

• Analysis of contours and boundaries analysis of objects

• Shape & positional invariance

• Contour enhancement for object identification

  • V1 is fundamentally important for conscious vision & perception

Higher visual areas in the cortex - V1

• processes raw visual input extracting key features from stimuli → relayed to higher brain areas → formation of conscious visual perceptions.

Higher visual areas in the cortex - V2

• integrates & combines visual information from neighbouring regions of V1

• responds to more complex patterns (corners/angles/textures)

• involved in stereoscopic depth perception, which allows the brain to perceive depth & three-dimensional structure

• provides feedback to V1 → refine the representation of visual features & contributes to the perception of complex visual scenes

Higher visual areas in the cortex - V4

• neurons respond to more complex stimuli (than V1/2)

  • Strong responses in V4 (red/orange strongest)

  • Strong responses in the anterior area of the inferior temporal cortex

object recognition

• Discrimination (<200 ms)

• Recognition of objects (changes in object position, size, viewpoint, and visual context)

• Categorisation

• Ventral cortical stream = critical for object recognition

• Number of neurons Colour – portion dedicated to central 10 deg of visual field Response latencies

• V1-V4: occipital lobes

• IT: inferior temporal cortex (temporal lobes)

Two visual streams in the cortex of primate/human brain

• Dorsal stream (where system): Interacting with the world (via V5/MT) 

  • If lesioned cant see where things are in the worlds 

  • in Parietal cortex

• Ventral stream (what system): Making sense of the world (via V4)

  • Is lesioned then can’t see colour

  • in Inferior temporal cortex

eye-hand coordination

• Guiding hand movements requires two processes 

  • Deciding which objects to interact with

  • Interacting with objects skillfully

• These processes require different type of information from both the dorsal and the ventral streamsK

Vision Key points

• Retina → 1st stages of visual processing 

  • (edge detection in visual scenes, edge enhancement in patterns,  filtering of spatial, wavelength, movement & directional information)

• Lateral inhibition in retinal cells is responsible for edge enhancement (and the Matchband effect)

• Edges & shadows provide context information about the spatial structure of objects or the spatial relationship between objects  

• Cones/rods converging on a bipolar cell form its receptive field, while cones/rods and bipolar cells converging to a ganglion cell form the ganglion cell’s receptive field.

  •  Receptive fields are larger in the periphery (blurrier vision due to lower acuity) and smaller in the fovea (helps to achieve highest acuity).  

• Some classes of bipolar and ganglion cells have a centre-surround receptive field. 

  • These can be ONcentre/OFF surround (lateral inhibition from receptors in the surround) or OFF-centre/ON-surround  (lateral inhibition from the photoreceptors in the centre of the centre-surround receptive field)

• Ganglion cells respond to ratios of light/dark (e.g. a small dot of light), but not to uniform illumination  

• P and M ganglion cells project to different layers in the LGN (and V1), and have different properties (receptive field sizes, conduction speed, acuity, presence/lack of colour sensitivity)  

• P and M ganglion cells project retinotopically to segregated layers in the LGN  

• V1 has columnar structure with neurons mapped & segregated in hypercolumns which combine orientation columns & ocular dominance columns for each part of the visual field. 

  • All neurons in an orientation column share the same preference for a particular orientation of a bar stimulus in their receptive field. 

  • Within hypercolumns orientation columns are found together if they receive input from either the left or right eye, thereby forming a pair of left-eye and right-eye ocular dominance columns in  each hypercolumn. 

• Simple cortical cells respond best to an edge or a bar of particular  width, orientation, and location in the visual field. 

• Complex cortical cells: respond best to a bad or  particular size and orientation anywhere within a particular area of the visual field