Lecture 5 - Sensory processing and memory

Key Topics:

  • Temporal Binding and Memory

  • The Auditory Event-Related Potential (ERP)

  • Adaptation and Memory in Auditory Cortex

  • The Mismatch Response

SECTION 1: Temporal Binding and Memory

Visual Feature Binding

  • The visual cortex is organized into multiple specialized areas, each processing specific visual features such as color, motion, and form.

  • For instance, the perception of a rolling red ball necessitates integration of various attributes (shape, color, motion) dispersed across these specialized areas.

  • Integration Process: The process of visual feature binding synthesizes information from these areas to create a cohesive perceptual experience.

Binding in Hearing

  • The auditory cortex comprises various areas whose functions remain incompletely understood, complicating the analysis of sound features such as pitch, loudness, and location, which are not distinctly represented in separate anatomical locations.

  • Temporal Binding: This is the mechanism by which auditory information is integrated over time, a critical function particularly during complex auditory tasks like speech perception.

Importance of Space and Time

  • Visual and Auditory Perception: Visual perception heavily emphasizes spatial context, while auditory perception is predominantly dependent on temporal context, with frequencies perceived as oscillations over time.

  • Visual Grouping through Binding: The Gestalt principles underline the visual grouping mechanisms, such as proximity, similarity, and figure-ground differentiation, that facilitate visual perception.

  • Visual information makes no sense without space

  • Auditory information makes no sense without time

  • Space is very important for visom

  • Time is very important for auditional processing, as it allows us to perceive sequences of sounds and understand speech patterns effectively.

  • Additionally, the integration of both spatial and temporal information is crucial for forming coherent memories, as it enables us to contextualize experiences and retrieve them accurately.

Visual grouping; gestalt principles

Gestalt principles - these principles explain how we naturally organize visual elements into groups or unified wholes based on certain characteristics, such as proximity, similarity, continuity, and closure, which play a significant role in our perception and memory formation.

  • Proximity: Elements that are close together are perceived as a group.

  • Similarity: Items that are similar in appearance are grouped together.

  • Continuity: Our eyes are drawn to continuous lines and patterns, aiding in memory retention.

  • Closure: The mind tends to fill in gaps in visual information, allowing us to perceive incomplete shapes as complete.

  • Figure-ground: This principle describes how we distinguish an object (the figure) from its background (the ground), which is essential for recognizing and remembering visual stimuli.

Auditory Grouping

  • Auditory scene analysis (ASA) is essential for distinguishing sounds from various sources to enable understanding and comprehension in auditory environments. Bregman (1990) emphasized that for meaningful perception, sounds must be effectively grouped into auditory streams.

  • Components of Sound Segregation: Sounds are organized based on frequency, timing, and location, necessitating processes like auditory stream segregation and integration to comprehend complex auditory scenes.

Auditory scene analysis

  • Components of sounds are segregated and grouped together according to proximity:

    • Frequency

    • time

    • Sound source location

  • Auditory stream segregation - the process by which the auditory system separates different sound sources in a complex auditory environment, allowing listeners to focus on specific sounds while filtering out others.

  • Auditory stream integration - the ability of the auditory system to combine different sound streams into a cohesive perceptual experience, enabling listeners to perceive and understand complex auditory scenes.

How do we hear sounds

  • Sounds are not heard in isolation

  • They are heard as members of streams

  • Streams are groups of sounds, extended in time

  • The way you hear a sound does not depend just on the sound

  • It depends on the historical context of the sound

  • This historical context includes previous experiences and knowledge, which can significantly influence how we perceive and interpret auditory information.

  • Additionally, the brain's ability to integrate these past experiences with current sensory input allows for more nuanced and meaningful interpretations of sounds in our environment.

Temporal Binding and Memory

The Brain’s Challenge

  • Spectrograms represent how sound frequency structures evolve over time, revealing critical insights into sound meanings and the cognitive processes involved in auditory perception.

  • Memory's Role: Without memory, the auditory system only retains segments of the spectrogram; integrating auditory information over time thus requires the activation of memory to facilitate temporal binding.

  • Sensory processing is not an isolated function but relies heavily on memory systems, as both auditory and visual perception integrate memory mechanisms for coherent understanding.

SECTION 2: The Auditory Event-Related Potential (ERP)

Event-Related Responses

  • The ERP is characterized by brain responses that are time-locked to specific stimuli, calculated by averaging responses across multiple presentations to mitigate the influence of ongoing background activity.

  • EEG (ERP) and MEG (ERF) are powerful tools for measuring these sensory processing responses.

Measuring Responses to Sounds

  • Event responses may be obscured by background noise; averaging trials helps illuminate the underlying event-related response by highlighting consistent responses to repeated sound stimuli.

Long-Latency Auditory ERPs

  • Significant components include P1, N1, and P2—representing the onset response sensitive to auditory features, influenced by factors such as learning and attention.

  • The N2 and P3 responses signify reactions to unexpected stimuli, revealing the complexity of the brain's auditory processing capabilities.

  • P1-N1-P2 = onset response, sensitive to stimulus features, and to effects of learning ,memory, and attention

  • N2 = surprising stimuli when these are consciously attended to

  • P3 = surprising stimuli,when these are task-relevant

SECTION 3: Adaptation and Memory in Auditory Cortex

Testing Memory in Sensory Systems

  • Memory in sensory systems elucidates how previous stimuli influence current sensory processing dynamics. The sensitivity to context can be evaluated by manipulating previous stimulation while keeping current stimuli constant.

  • Context sensitivity = the ability of the sensory system to adjust its response based on prior experiences or exposures, thereby enhancing or diminishing the perception of new stimuli.

    • This phenomenon is particularly evident in auditory processing, where sounds previously encountered can alter the perception of new auditory inputs, leading to either heightened awareness or selective attention.

Adaptation and Facilitation

  • Repetition of stimuli leads to response dampening, a phenomenon known as adaptation, whereas presenting a new stimulus after a series of similar ones can enhance responses—a process referred to as forward facilitation.

  • Foreward masking = the phenomenon where a stimulus interferes with the perception of a subsequent stimulus, often resulting in a diminished response to the latter.

  • Foreward facilitation = the process by which the presentation of a novel stimulus following a series of similar stimuli enhances the perceptual response, allowing for improved detection and discrimination of stimuli.

  • This interplay between adaptation and facilitation is crucial for understanding sensory processing, as it illustrates how our perceptual system adjusts to both familiar and novel stimuli.

N1 Adaptation

  • The N1 component represents a dominant ERP response within the first 200 milliseconds, peaking around 100 milliseconds, indicative of the widespread activation across the auditory cortex during sound processing. Subsequent presentations of the same stimulus result in attenuated responses, exemplifying the effects of sensory adaptation.

  • 1. Most prominent ERP response in first 200 ms

  • 2. Peaks at 100 ms

  • 3. Represents wide spread activation of auditory cortex

  • When you repeat a stimulus (here, after 1 s) ,the response to the repeated stimulus is small. This is called adaptation


SECTION 4: The Mismatch Response

Mismatch Negativity (MMN)

  • MMN is characterized by a larger ERP response to unexpected stimuli compared to standard, predictable ones, reflecting the auditory system's discriminative capabilities.

  • This response involves memory tracking mechanisms that facilitate the encoding of probabilities and unexpected values in auditory perception.

  • The standard elicits an adapted response

  • The deviant produces a larger ERP than the standard

  • The difference is called the MMN

  • Interpretation: the auditory system is

    • Discriminating Standard from Deviant

    • Keeping track of their probabilities & surprise value

    • ››This requires ”memory

Oddball Paradigm

  • In experiments employing the oddball paradigm, a standard stimulus elicits an adapted ERP response, while a deviant stimulus generates a pronounced ERP response (MMN).

  • Standard and Deviant can differ along any stimulation feature:

    • Frequency

    • Intensity

    • Sound duration

    • Sound source location

    • Speech sound identity

    • Tone sequence structure

Significance of oddball paradigm

  • Role of attention in auditory perception

  • Impact of sensory integration on memory encoding

  • Neural mechanisms underlying sound processing

  • N1 adaptation reflects sensory memory

  • MMN reflects stimulus discrimination

  • MMN is a response to surprise –part of the orienting reflex

Significance of MMN

  • MMN signifies the brain's capacity for stimulus discrimination and highlights the role of sensory memory in perceiving auditory events.

  • Involuntary physiological reactions to unexpected stimuli (e.g., skin conductance, heart rate alterations) signal attentional shifts, contributing to the understanding of cognitive processing in response to sound.

The Orienting Response (OR)

  • The OR is a critical cognitive mechanism observed in responses to infrequent stimuli, where surprise results in notable physiological reactions, such as eye movement changes.

  • Infrequent stimuli cause surprise

  • Skin conductance response

  • Heart rate changes

  • Eye movements

  • Halting of ongoing activity

  • MMN

  • Involuntary reorienting ofattention

Predictive Context in Auditory Processing

  • Auditory processing entails using previous experiences to construct auditory expectations, indicating a top-down process where higher cortical predictions interact with incoming sensory data.

  • The brain continually assesses the level of match between expectations and sensory input; prediction errors guide modifications in perception and attentional focus.

  • This dynamic interaction allows for more efficient processing of sounds, as the brain prioritizes stimuli that deviate significantly from expectations, thereby enhancing our ability to detect important auditory signals in complex environments.

  • In this way, predictive coding not only influences how we perceive sounds, but also shapes our memory of auditory events, enabling us to remember and recognize familiar sounds more effectively.

Predictions help the brain act effectively

  • 1) We use our previous experience to generate predictions about the future, creating a model

  • 2) This “top-down” prediction travels from higher cortical areaslike frontal cortex

  • 3) “Bottom up” information comes from sensory signal e.g. V1/A1

  • 4) The brain evaluates how well the two signals match

  • 5) Sends prediction error:

    • 1) accurate = small prediction error, signals to trust model

    • 2) Inaccurate = large prediction error, can signal change or“don’t trust your model”

      This can affect the way we perceive the sensory information

Predictability enhances saliency

General Summary

  • Sensory integration over time, termed temporal binding, is a fundamental aspect of coherent perception, underscoring the importance of memory in linking disparate sensory inputs.

  • Key evidence supporting this notion includes N1 response adaptation and MMN effects within the auditory cortex.

  • The next lecture will introduce topics on brain plasticity and the mechanisms underpinning memory production.