Multisensory Processing Study Notes

Definition: Multisensory processing refers to the ability of the brain to integrate information from different sensory modalities (sight, sound, touch, etc.) to create a coherent understanding of the environment.

Old View: Traditionally, it was believed that information from each sensory system is processed independently.
Emerging View: Recent research suggests that sensory systems influence one another at multiple stages of processing.

Definition: These are the associations between sensory experiences where one sensory modality can influence the perception or experience of another modality.
Example Concepts:
- Kiki-Bouba Effect: A classic example studied by Kohler (1929) and later by Ramachandran & Hubbard (2005) that illustrates how people are likely to associate certain shapes with specific sounds.
- Kiki: A jagged, spiky shape.
- Mooba: A round, smooth shape.

Shared Neural Code: High-intensity stimuli lead to rapid firing rates in neurons, making loud sounds comparable to bright lights.
- Example Saturation: Labels like “heavy” and “strong” can evoke similar multisensory feelings.
Similarities Across Sensations: Certain adjectives (e.g., “soft,” “warm,” “sweet”) are generally perceived as physically pleasant, leading to stronger associations compared to negative descriptors (e.g., “sharp,” “cold,” “bitter”).

Packaging Shapes and Colors:
- Figure 1: Angular red shapes are a key feature in logos of various beverages.
- Brands like San Pellegrino, Apollinaris, and Heineken utilize these geometrical shapes for brand recognition (Spence, 2012).

Definition: The process by which one sensory experience is influenced by another.
- Example: The McGurk effect demonstrates this integration: when people see a video of someone saying "ga" but hear audio of "ba," they often perceive it as "da." This demonstrates the influence of visual information on auditory perception.
Other Examples of Multisensory Integration:
- Sensory modalities often integrate in various contexts, such as:
- Hearing + Vision
- Flavor Combination: Taste is influenced by smell, touch, vision, and hearing.
- Vision + Vestibular + Proprioception: These modalities work together to assist with balance and spatial orientation.

Redundancy: Multiple senses provide overlapping information in case one sense is impaired.
Cross-Checking: Inconsistencies between different senses can hint at errors or anomalies in perception.
Complementarity: The integration can lead to enhanced memory formation concerning the sensory experiences.
Generalization: For example, if tasting a white berry proves bitter, one might avoid another white berry preemptively based on this experience.

Definition: Synesthesia is described as “an involuntary joining in which the real information of one sense is accompanied by a perception in another sense” (Cytowic, 1989).
Types of Synesthetes:
- Projectors: Experience the concurrent sensory perception in physical space.
- Associators: Experience concurrent perceptions in their ‘mind’s eye.’

Grapheme-color Synesthesia: Letters or numbers are associated with specific colors.
- Consistency: Colors reported are highly consistent (80-100%) within individuals.
Lexical-gustatory Synesthesia: Words trigger specific flavors; e.g., experiencing the word “cold, hard bacon” as a flavor which can be either pleasant or unpleasant.
Ordinal-linguistic Personification: Letters or numbers may be personified with traits, colors, gender, or personalities.
Number Form Synesthesia: Temporal perceptions like days or months are spatially organized.
Others: Include auditory-tactile synesthesia, pain-color synesthesia, among others.

Frequency: Occurrence estimates range from 0.2% to 4.0% of the population.
- More likely to be female and left-handed.
Genetic Factors: Presence of synesthesia is inherited rather than the specific type.

Synesthetes often experience faster processing times in sensory tasks compared to non-synesthetics, demonstrated by the pop-out effect and mathematical interference tasks.
Recent neuroimaging studies have identified specific brain regions associated with sensory processing, such as:
- Areas processing color (V4), letters, and their interactions (Hubbard et al., 2005).

Philosophical Implications: The concept of no sensory modality existing in isolation emphasizes the interconnectedness of human perception. Multisensory processing reveals a fundamental aspect of how we experience the world, showcasing that our senses do not operate in solitude but rather in a rich tapestry of experiences that inform one another.