PSYB55 Lecture 04: Visual Processing, Synesthesia, and Experimental Design

LECTURE 04: Visual Processing, Synesthesia, & Experimental Design

Lecture Objectives

This lecture aims to provide a comprehensive understanding of visual processing, from initial transduction to complex cortical activity, explore cross-sensory phenomena like synesthesia, and apply principles of experimental design compatible with neuroimaging:

• Visual Pathway: Describe the flow of visual information from the eye's transduction to the primary visual cortex (V1) in the occipital lobe, and how disruptions along this pathway can impact vision.

• V4 & V5: Understand the engagement properties of visual areas V4 and V5 using both functional neuroimaging and patient studies.

• Information Complexity: Articulate how the complexity of information changes along processing pathways towards the temporal lobes.

• Synesthesia: Describe synesthesia and identify how structural and functional neuroimaging can elucidate its mechanisms.

• Experimental Design: Apply understanding of conceptual and operational definitions to create tasks suitable for neuroimaging.

From Sensation to Perception

Visual processing transforms raw sensory input into meaningful perception. The journey begins with the eye's gross anatomy.

Gross Anatomy of the Eye

Key components include:

• Cornea: Transparent outer layer.

• Lens: Focuses light onto the retina.

• Retina: Contains photoreceptors (rods and cones) for light transduction.

• Sclera: White outer layer.

• Optic Nerve: Transmits visual information to the brain.

Retinal Implant Devices

These devices exemplify how disruption along the visual pathway can be addressed. For patients with degenerated photoreceptor layers, subretinal implants can provide light-sensitive micro-photodiodes, connected via cables to an amplifier, power supply, and transmitter/receiver, allowing blind patients to perceive letters and words. This highlights the importance of the initial transduction step in the eye.

Primary Projection Pathways of the Visual System

Light from the visual field enters the eye and is processed along specific pathways:

1. Left Visual Field: Projects to the right hemisphere.

2. Right Visual Field: Projects to the left hemisphere.

3. Optic Chiasm: Where nasal portions of the optic nerves cross over, ensuring that information from each visual field goes to the contralateral hemisphere.

4. Lateral Geniculate Nucleus (LGN): A thalamic relay station for visual information, receiving input from the optic chiasm.

5. Superior Colliculus: Also receives visual input and is involved in eye movements.

6. Optic Radiation: Fibers from the LGN project to the primary visual cortex (V1).

7. Primary Visual Cortex (V1): Located in the occipital cortex, this is the first cortical area to receive visual input. It maintains a retinotopic map, meaning adjacent points in the visual field are processed by adjacent points in V1.

Consequences of Disruption in the Visual Pathway

Disruptions at various points can lead to specific visual deficits:

• Scotoma: A blind spot within the visual field caused by localised injury to the retinotopic map on the primary visual cortex. The scotoma's location corresponds directly to the damaged cortical area.

• Cortical Blindness: Complete loss of vision due to extensive damage to V1, despite intact eyes.

• Hemianopias: Visual field deficits affecting half of the visual field, typically caused by damage further along the visual system, such as posterior to the optic chiasm. Examples include right-sided circumferential blindness due to retrobulbar neuritis.

Congenital Blindness and Brain Reorganization

Congenital blindness (due to eye dysfunction) can lead to fascinating brain reorganization. Studies like those by Ptito et al. ($2008$) investigate how the brain adapts and reassigns functions in the absence of typical visual input, highlighting neural plasticity.

V4: The Color Processing Centre

Discovery and Function

• Discovery: Zeki et al. ($1991$) were instrumental in identifying V4 as a key area for color processing. This was achieved by comparing brain activity during tasks involving color perception versus non-color tasks.

• V4 Complex and Color Constancy: Bartels & Zeki ($2000$) demonstrated V4's role in color constancy. Color constancy refers to our ability to perceive an object's color as stable despite changes in lighting conditions. The V4 complex adjusts for varying illumination to maintain a consistent perceived color.

• Stimuli Coloration and Selective Attention: Research also shows that V4 is involved in processing stimuli coloration, even when attention is selectively directed to a specific hemifield, further isolating its role in color computation.

Cerebral Achromatopsia

• Definition: Patients with cerebral achromatopsia experience a profound loss of color vision due to brain damage, typically in V4, while their eyes remain functional. They perceive the world in shades of grey.

• Distinction from Prosopagnosia: It's crucial to differentiate achromatopsia from other visual deficits. For example, a patient can have achromatopsia (impaired color perception, face perception OK) but not prosopagnosia (inability to distinguish faces, color perception OK). This double dissociation, as shown by Bouvier & Engle ($2006$), indicates separate neural substrates for these functions.

V5: The Motion Processing Centre

Discovery and Function

• Discovery: Zeki et al. ($1991$) also identified V5 (also known as MT, medial temporal area) as the brain's motion processing center. This was achieved by contrasting brain activity during flickering stimuli with pixel movement versus static images.

• V5 Activation: Watson et al. ($1993$) provided further evidence, illustrating V5 activation overlaid on sMRI scans, showing specific regions for motion perception.

• Processing Illusory Motion: V5 is not only active during real motion but also during illusory motion. Zeki, Watson & Frackowiak ($1993$) observed V5 activation when participants perceived movement in a static image (illusory motion) compared to a control, suggesting its role in the perception, not just detection, of movement.

Inducing Akinetopsia with Transcranial Magnetic Stimulation (TMS)

• Cerebral Akinetopsia: This condition involves the inability to perceive motion, seeing the world as a series of still frames. It can result from brain damage to V5.

• TMS Experiment: Beckers & Zeki ($1995$) demonstrated that transcranial magnetic stimulation (TMS) applied over V5 can temporarily induce akinetopsia in healthy individuals. TMS disrupts normal V5 function, while TMS over non-V5 prestriate cortex or striate cortex (V1) does not produce the same effect, highlighting the specificity of V5 for motion processing.

Specificity of V5 Neurons and Visual Pathways

• Macaque Visual System: Research on macaques provides insights into the organization of visual processing. Initial visual input goes from the Retina to the LGN to V1, then diverges into two main streams:

  • Dorsal Stream (Parietal Cortex): Involved in spatial awareness and 'where' processing. Key areas include V2, V3, MT (V5), MST, PO, VIP, and LIP.

  • Ventral Stream (Inferotemporal Cortex): Involved in object recognition and 'what' processing. Key areas include V2, V3, V4, TEO, and TE.

• Receptive Fields: The receptive field of a neuron is the area of the visual field that, when stimulated, causes a change in the firing rate of that neuron. Information complexity increases along these pathways:

  • V1 Receptive Fields: Small, selective for simple features like oriented bars.

  • V4 Receptive Fields: Larger than V1, selective for more complex features like shapes and colors.

  • TEO Receptive Fields: Even larger, integrate information about parts of objects.

  • TE Receptive Fields: Very large, responsive to whole objects, often invariant to position and size, crucial for object recognition. This progression highlights how raw visual input is gradually transformed into complex percepts through hierarchical processing.

• Akinetopsia Lesion Reconstruction: Stevens et al. provided a reconstruction of a lesion producing severe akinetopsia, pinpointing affected areas such as the superior temporal gyrus (STG), medial temporal gyrus (MTG), lateral occipital gyri (LOG), angular gyrus (AG), and supramarginal gyrus (SMG), further supporting the distributed nature of motion processing involving V5 and related areas.

Synesthesia and Cross-Sensory Processing

What is Synesthesia?

• Definition: Synesthesia is a phenomenon where one sensory experience automatically and involuntarily triggers another distinct sensory experience.

• Characteristics:

  • Automatic and Involuntary: The cross-sensory experience is not chosen or willed.

  • Non-transitory: It's a consistent way an individual processes the world, not a temporary state.

  • Prevalence: Estimates vary widely, but it is known to be more common in females than males, and in left-handers than right-handers.

Types of Synesthesia

Many types exist, including:

• Grapheme-Color Synesthesia: Letters or numbers (graphemes) automatically evoke the perception of specific colors.

• Chromesthesia: Sounds automatically trigger color perceptions.

• Spatial Sequence Synesthesia: Numbers, days of the week, or months are perceived as having specific locations in space.

• Auditory-Tactile Synesthesia: Certain sounds induce tactile sensations on the body.

• Mirror-Touch Synesthesia: Observing someone else being touched leads to a sensation of being touched in the same location on one's own body.

Mechanisms of Synesthesia

Structural and functional neuroimaging can help elucidate the neural mechanisms:

• fMRI and DTI in Grapheme-Color Synesthesia: Rouw & Schulte ($2007$) used fMRI and Diffusion Tensor Imaging (DTI) to investigate grapheme-color synesthesia. They found structural differences (e.g., increased white matter integrity) and functional hyperconnectivity (e.g., stronger activation or connections) between brain regions responsible for color processing (V4) and grapheme processing (visual word form area). This suggests that synesthesia might arise from atypical neural connections or cross-activation between sensory areas that are typically more segregated.

Multisensory Integration: The McGurk Effect

• Definition: The McGurk effect is a perceptual phenomenon that demonstrates an interaction between hearing and vision in speech perception. What we see can influence what we hear.

• Example: When a person hears the auditory syllable '/ba/' but sees the visual articulation of '/ga/', they often perceive the syllable '/da/'. This illustrates how the brain integrates conflicting sensory input to form a coherent, albeit sometimes illusory, perception, showcasing the brain's inherent multisensory processing capabilities even in non-synesthetic individuals.

Designing Experiments: Part II

Fundamentals of Experimental Design

Carefully designed experiments are crucial for cognitive neuroscience research, especially those compatible with neuroimaging:

• Importance of Studying Constructs: Researchers must articulate why a given construct (e.g., working memory load, pressure response) is an important target of study.

• Conceptual Definition: A clear, theoretical definition of the construct being investigated.

• Operational Definition: A precise, measurable definition of how the construct will be manipulated or measured in the experiment.

• Many Trials and Participants: It's important to study phenomena with many trials (to increase statistical power and reliability) and many people (to ensure generalizability of findings and account for individual variability).

Experiment Example 1: Manipulating Working Memory (WM) Demands

• Construct: Working memory (WM) load.

• Conceptual Definition: The amount of information that can be held and manipulated in mind over a short period.

• Operational Definition (WM Manipulation): Varying the number of items to be remembered.

• Task Paradigm (Trial Flow):

  1. GET READY! (Attention cue)

  2. STIMULUS (e.g., $500$ ms): Presentation of a sequence of letters (e.g., "X O Q").

  3. DELAY (e.g., $3500$ ms): A period where the letters must be maintained in WM.

  4. PROBE (e.g., $7500$ ms): Presentation of a single letter, asking if it was part of the original sequence (YES: left button; NO: right button) with a $2000$ ms response window.

• Manipulating WM Demands:

  • Lower WM Demands: Presenting a short sequence (e.g., "X O Q").

  • Higher WM Demands: Presenting a longer sequence (e.g., "P T Q M G I").

• Considerations: Does the type of stimuli matter (numbers, letters, words, pictures)? This impacts stimulus encoding and the cognitive processes engaged.

Experiment Example 2: Manipulating the Degree of Pressure

• Construct: How individuals respond to higher pressure situations.

• Conceptual Definition: The psychological or physiological state induced by perceived high stakes, evaluative threats, or demanding circumstances.

• Operational Definition (Pressure Manipulation): Varying the social or performance demands of a hypothetical scenario.

• Task Paradigm (Trial Flow): Similar to WM, with STIMULUS (read scenario), DELAY (consider feelings), PROBE (would you do it?).

• Manipulating Pressure:

  • Lower Pressure: "Suppose you're asked to tell your best friend how your day went yesterday…"

  • Higher Pressure: "Suppose you're asked to give a speech on your favorite Cog Neuro method in $10$ minutes…"

• Considerations: Is this a fair way to capture the concept of interest? Could an actual performance task be more effective than a hypothetical scenario? What alternative ways exist to induce and measure pressure?

Practice: Creating a Cognitive Neuroscience Experiment

For a chosen process (e.g., Lie Detection, Managing Distractions):

1. Argument for Importance: Justify why this construct is a significant area of study.

2. Conceptual Operational Definition: Provide a clear conceptual definition of the construct.

3. Working Operational Definition of Manipulation: Define how the construct's level will be varied.

4. Task & Control Condition: Design an experimental task that engages the process and a suitable control condition that differs only in the construct of interest.

5. Relevance of the Subtraction Method: Explain how subtracting brain activation during the control condition from the experimental condition isolates the neural activity specific to the process being studied. For example, if measuring lie detection, compare brain activation during lying (experimental) vs. truth-telling (control) to pinpoint regions involved in deception. This method assumes pure insertion, that additional cognitive processes can be added without affecting the duration or nature of the existing ones.

Relevant UTSC Psychology Courses and Research Opportunities

• Courses: PSYB51 (Introduction to Perception), PSYC51 (Cognitive Neuroscience of Vision), PSYC56 (Music Cognition), PSYD51 (Current Topics in Perception), PSYD54 (Current Topics Visual Recognition).

• Research: Opportunities to get involved in perception research at the University of Toronto by contacting UTSC and UTSG researchers.