sensory modalities

colour vision

• Humans do not see colour in dim light 

  • Colour vision = bright light -> 3 type of cones

  • No colour vision = dim light -> 1 type of rods

Ishihara test

•  a psychophysical test that is clinically used to identify colour deficiencies in humans

colour vision in humans

• cannot see colour in dim light

  • colour vision → bright light = 3 types of cones (ospins)

    • requires strong stimulation + rapid temporal response

  • no colour vision → dim light = 1 type of rod

    • highly sensitive → weak stimulation & slow temporal response

Properties of the human photopic visual system

• cones

• approximate 4 million per eye

• 3 classes of cone ospin photopigments

  • low sensitivity

    • needs relatively strong stimulation

    • used for everyday vision

  • located in fovea

    • present less densely throughout retina

  • rapid temporal response

Properties of the human scotopic visual systems

• Rods

• 100 million receptors per eye 

• Rhodopsin photopigments

• High sensitivity 

  • Can be stimulated by weak light intensity 

  • Used for night vision 

• Located outside fovea 

• Receptive field size larger so acuity is lower 

• Slow temporal responses

colour deficiencies

• mild → trichromatic colour vision

• colour blindness (severe) → dichromatic colour vision

  • X chromosome defects are frequent

  • more men affected → only have 1 X

most mammals have dichromatic colour vision

• Most mammals are dichromats & lack the m-cone

  • Have s and L cone 

• Trichromats (human, few old world primates) discriminate more colour than dichromats 

  • See using 3 cones

• Marine mammals do not see colour = only have L-cone 

• Tetrachromacy refers to seeing colours through four cones which allows them to see more colours

Types of cones

• S-cones 

  • Short-wavelength sensitive receptor

  • 420nm

• M-cones

  • Medium-wavelength sensitive receptors

  • 530nm

• L-cones

  • Long-wavelength sensitive receptor 

  • 560nm

what is colour

• Light stimulus → Illuminating light Reflection

• Colour is not part of the physical reality → subjective experience of physical reality 

  • result of neural activity in response to light stimuli 

• Light stimuli can be inherently ambiguous at photoreceptor level

• Spectral reflectance of different coloured object surfaces

  • Light that is reflected 

  • Measured using photospectrometer

Colour coding & perception

1. small optical projections of scattered light of different wavelengths

2. signals from photoreceptors enable comparisons of wavelengths in visual field

3. helps in colour detection, colour discrimination & identifying between objects

colour vision pathway

• s, m & L-cones in retina & bipolar cells

• colour coding P ganglion cells in retina & P-cell layers in LGN

• colour sensitive neurons

colour coding from retina to V1

• P- and M- ganglion cells project to different layers in the LGN

• P-ganglion cells

  • project to Parvocellular layer in LGN

  • small RFs, slower conduction speed, high acuity, poor response to transient stimuli, colour-sensitive

•  M-ganglion cells

  • project to Magnocellular layer in LGN

  • large RFs, higher conduction speed, sensitive to motion, low acuity, no colour discrimination

segregation in processing of visual information

• Although both P- and M-ganglion cells receive input from the same photoreceptors, they maintain their segregation by projecting to different layers in the LGN.

  • segregated projections from retina to LGN are also retinotopic

P-ganglion cells

• project to Parvocellular layer in LGN

• small receptive fields

• slower conduction speed

• high acuity

• poor response to transient stimuli

• colour-sensitive

M-ganglion cells

• project to Magnocellular layer in LGN

• large receptive fields

• higher conduction speed

• sensitive to motion

• low acuity

• no colour discrimination.

Hearing guides behaviours

• to detect and discriminate locations and movement of sounds sources 

  • Spatial orientation 

  • Echolocation (bats, whales, humans) 

  • Auditory communication & language

Airborne sound waves

• sound → pressure waves, movement of air particles set in motion by vibrating

• Measurements to characterise an auditory sound stimulus

  • Sound frequency (Hertz (Hz) → reciprocal of wavelength & perceived as pitch/tone

  • Amplitude (perceived as loudness) → relative strength of wave as transmitted vibration

sound stimuli are transformed to vibrations in the ear

• ear does not spatially map the locations of sound and only sorts sounds by wavelengths via tonotopic mapping in the inner ear

• Air-borne sound waves and/or bone-conducted sound vibrations impinge on the tympanum and middle-ear bones which then vibrate accordingly 

• Vibrations are amplified by the middle ear bones in order to transmit the stimulus to the oval window of the cochlea (inner ear)

tonotopic arrangement of hair cells

• sound waves cause vibrations in the hair → amplified & transmitted to inner ear

• a mechanic receptor 

• low pitch will be closer to apex of cochlea

• high pitch will be closer to base of cochlea

Stereocilia (stiff hair) help to stretch open the ion channels

• Bending of the stereocilia (input zone)

• Opening of nonselective ion channels that allow influx of K+ and Ca2+ ions

• Depolarisation of the hair cell → opens voltage-gated Ca2+ channels in the base of the hair cell (output zone)

  • Neurotransmitter is released to excite afferent auditory interneurons

Auditory tuning curves & behavioural audiograms

• auditory tuning curves describe the frequency selectivity of auditory neuron

• behavioral audiograms provide a comprehensive assessment of an individual's hearing sensitivity across different frequencies

• Auditory interneurons → help us understand preferred frequency

Auditory pathway

• receptor → primary cortex

• Most projections from the cochlea to the contralateral cortex occur via the cochlear nerve and cochlear nuclei.

• Each superior olivary nucleus in the brainstem receives inputs from both cochlear nuclei

  • serving as the 1st stage of binaural coding for the spatial location of a sound source.

• Further tonotopic (frequency-based) & spatial mapping occurs in the inferior colliculi

  • located in the dorsal midbrain

• medial geniculate nuclei of the thalamus also contribute to tonotopic &spatial mapping.

• Tonotopic & spatial mapping processes culminate in the primary auditory cortex,

  • where neural representations of sound frequency and spatial location are further refined & processed.

Cross-modal brain plasticity 

• Coding of sensory information is similar across modalities

  • Vision

    • Colour, Shape, Motion

      • Light stimuli

      • Eye photoreceptors

  • Hearing

    • Pitch/Tone, Sounds (music, vocalisations), Noise 

      • Sound stimuli

      • Ear hair cells

  • Touch

    • Pressure, Vibration, Tension

      • Mechanical stimuli

      • Skin receptors

  • Smell

    • Odour 

      • Airborne molecules

      • Nose chemoreceptors

• labelled line

Multimodal processing and integration

• Better performance if binaural stimulation (both ears listen)

  • Also demonstrated in other sensory modalities (e.g. fast visual sequence)

• Perceptual segregation of stimuli: brain filters and predicts through uni/multimodal bottom-up processing and integration but exerts also top-down control to affect various stages and connections during sensory processing

• Some stimuli can be more salient, e.g. strongly contrasting colours or semantic ones, such as one’s own name, and therefore be less suceptible to interference

Cocktail party effect

ability to drive attention towards one stimulus filtered out from the noisy environment

Cross-modal neural reorganisation following sensory loss

• Recruitment of visual cortex in blind subjects

  • Brain uses pathways interchangeably to help

Multimodal integration

Covert direction of (selective) attention

• Different from vigilance (global state of alertness) 

• Overt direction of attention

  • gaze centred on area of interest coincides with the visual information selected for attention (e.g. when reading a text) 

• Covert attention

  • gaze fixates on location in visual field whilst information from another part is selected for attention 

• Child blindness

  • Magicians are masters in misdirecting and splitting attention, introducing uncertainties or multilevel tasks with multimodal sensory stimulation in addition to social cues 

• Eye tracking 

  •  gaze directed to face (when asked a question by magician) resulted in more change blindness

Multimodal processing and integration

• Cocktail party effect

  • ability to consciously drive attention towards one stimulus filtered out from the noisy environment

• Better performance if binaural stimulation (both ears listen) 

• Also demonstrated in other sensory modalities (e.g. fast visual sequence)

• Perceptual segregation of stimuli: brain filters and predicts through uni/multimodal processing and integration but exerts also top-down control to affect various stages and connections during sensory processing

• Some stimuli can be more salient, e.g. strongly contrasting colours or semantic ones, such as one’s own name, and therefore be less susceptible to interference

key points sensory modalities

• Colour vision relies on three cone types in the retina, expressing S, M, or L opsins sensitive to various wavelengths.

• Neurons in the P pathway and ventral stream (ganglion cells, LGN, V1, V2, V4) process color and maintain color constancy.

• Animals have no color vision, dichromatic (mammals), trichromatic, or tetrachromatic color vision.

• Humans typically have trichromatic vision, but deficiencies arise from missing or shifted opsins

• Defective opsin genes, often on the X chromosome, lead to more common color deficiencies in males.

• Light and sound propagate as waves with differences in frequency and intensity.

• Animals locate sound sources by comparing information processed by both ears.

• Sound vibrates ear structures, opening ion channels in hair cells with stereocilia.

• Inner hair cells release glutamate to excite first-order auditory interneurons, transmitting signals to the cochlear nucleus and brainstem.

• Signal coding is modulated by outer hair cells and efferent interneurons.

• The auditory pathway has parallel and serial connections, similar to vision.

• Audiograms enable cross-species comparisons to understand hearing adaptations to tasks and ecological needs.

• Various brain areas receive input from different sensory modalities.

• Multimodal integration can generate unique perceptual qualities or diversify salient cues.

• Attention processes aid the brain in prioritizing information.