Module 2 - nuerolinguistics

Auditory Localization ex: owl

Owl flies silently - How does it locate mice under snow auditorily? - position, direction, distance of sound source

Computational Level goal: Locate mouse 

Input of computation: location  

Output of computation: sound

Location of object - visual field

• Large difference - close 

• Small difference - far away 

Each nueron - 2 inputs 

• One connecting one way 

• Sound - hits one ear before the other 

Sound activation - every line passes nueron down 

• Signal comes from left ear then right ear hears it later - passes nueron up 

cochlea configuration

• Sound/frequencies travels through cochlea  

• Base of cochlea (5mm) - narrow - tuned for high frequencies (8000 Hz)

• Apex of cochlea (30mm) - wider - lower pitch frequencies (200 Hz)

• Amplifies version of sound

tonotopy

cortical spatial code for frequency information/pitch

• frequency mapped to spatial gradient in the brain 

  • Sounds with adjacent frequencies activate adjacent neurons to each other

• spatial code created in cochlea perceived in auditory cortex 

• Could also say neurons in primary auditory cortex have different frequencies

Speech perception

Acoustic signal →

nuero auditory represent tion (nuerogram) →

Nuero phonological representation   (phonological sketch) - figure out what you are hearing   Sound sometimes overlap - bait vs bet

Lexical item  understand words, morphemes

Cochlea

coiled mechanism containing organ that converts sound waves to electrical brain impulses

• thousands of tiny hair cells move when the fluid vibrates

• processes then send action potentials along nervous system

• Spatial codes - hair cells at diff locations moved by diff frequencies

Cochlea layout/positioning

Cochlea goes from narrow to wide 

• Diff frequencies emphasized at different parts 
  
  

Cochleagram

neural responses that represent sound at the brain’s periphery 

• maps how the inner ear processes frequencies

Auditory cortex -

-where patterns encoded with fine degree of detail

• X & y axis that encode spectral & temporal details of stimuli

Phonemes

phonemes - speech sounds that words are built of

• physical description of a speech sound

• Physically real 

• Sounds of thing in environment 

ex [p]

• the singular sound [p] of something in an environment

Phone

• Mental description of a speech sound 

• Mental/ symbolic representation- stands for something

• Features define the phoneme

ex /p/

• mental representation/symbol of p

Ganong effect different from McGurk 

• Shows that continuous acoustic info not enough to explain brain recognition of phonemes & words 

• person's knowledge of vocabulary (top-down processing) changes how they hear an ambiguous speech sound (bottom-up processing

• if hearing ambigious speech sound halfway between /g/ and /k/) followed by -ift, people hear gift (a real word) rather than kift (a nonsense word).

diff from Mcgurk because its top down lexical knowledge vs visual cues

• Was found that electrodes at different sites respond to different phonemes 

• Nemo Mesogarani- shows that populations of neurons in STG have phonemic receptive fields 

  • 1 electrode may be sensitive to class of stop consonants, not specific phones

Criteria for being a phoneme

1 - Minimal pairs - Does it change the meaning 

Ex:    

• /kæt/

• /gæt/

2 - Rule application - Can it be applied to multiple words? 

Ex: 

• in+possible

• In+credible

Distinctive features

what phonemes are built from

Phonemic receptive fields

• areas of neurons in the auditory cortex

• tuned to select, process, and differentiate phonemes.

Phonemic receptive fields ex - Mesgorani et al (2014)

Different groups of phonemes (dipthongs + vowels, nasal, plosives) have different activations of neurons - phonemic receptive fields

Temporal windows of integration - short

25-40 ms- left hemisphere response

• make sense of the structure properties of diff phonemes

• Phonemes, features 

• auditory input: better speech comphenesion- fine structure preserved 

• words- no fine structure at all - can still understand 

Temporal windows of integration - long

200-300 ms- right hemisphere response 

• understand properties of syllables & other structures in temporal envelope, audiovisual integration 

• identify temporal properties of syllables like stress, intonation, prosody,

Asymmetric Sampling in Time (AST) hypothesis

• study Zeng et al (2002)

• listeners perform better when

when fine structure information goes to left hemisphere (phonemic feature length) & envelope info goes to right hemisphere (syllable length) 

- auditory input: better speech comprehension (higher performance) when speech envelope preserved 

- words - no envelope- impossible to understand

Lateralization

Tuning function in lateral Heschl’s gyrus 

• Explaining in cases of aphasia - intact prosody because it is located in right hemisphere 

Analysis by synthesis ex - phonemic receptive fields

• Takes info in a neurological way and maps it to cochlea 

Generally goes forward 

•  take information about sensory event and you build upwards - what do you think is true about phonology of word - can build upwards in this 

Top down thingy 

• Info filters backwards and can influence what you hear/supposed to hear  

Breaking something down to understand it 

• Break it down by building it up 

• Figure out what words are by figuring out what parts are made of - figure out what parts are made of by figuring out what words are 

How is sound represented in auditory cortex?

Look at:

1. tonotopy - cortical spatial code for sound/frequency information

• how frequency is mapped to spatial gradient in the brain

2. receptive field of neuron - connection between a certain stimulus and the response of a given neuron

Speech envelope tracking

how the brain's cortical activity synchronizes with the slow, rhythmic amplitude fluctuations (the overall volume or energy shape) of spoken language over time

• The Speech Envelope: This is the slow-moving outer contour of a sound wave

• Neural Synchronization: When people listen to speech, neurons in the auditory cortex fire in phase with these slow acoustic peaks and valleys. [1, 2]

• Role in Comprehension: strength and precision of this tracking often correlate with how well a listener understands speech, handles background noise, or processes language.

Analysis by synthesis

acoustic info →  nueral representation of Sound (neurogram) → phonological representation 

Analysis aided by synthesis of speech output - based on top down knowledge

Speech envelope

outline of the peaks (loud moments) and troughs (soft moments) in waveform - gives speech patterns of syllables & stress 

Mapping phonological features in the brain - Multi-voxel pattern analysis 

• Used to understand the neural representations that matter to a particular set of voxels in the brain 

• Manner and place of articulation of voicing 

• Whats going on in phonological sketch - underspecification 

Formant

• a peak in frequency energy - amplified frequency representing specific sound in vocal tract 

•  F1 (First Formant): Inversely related to vowel height. Open vowels (like /a/) have a high F1 frequency, while close or high vowels (like /i/ or /u/) have a low F1 frequency.

• F2 (Second Formant): Related to the backness or frontness of the tongue. Front vowels (like /i/) have a high F2, while back vowels (like /u/) have a low F2.

• F3 (Third Formant): Helps define speaker identity, voice quality, and finer acoustic characteristics.

Speech envelope

• Outline of speech 

• Don’t have high level information 

• captures Rhythm, prosody, vowels, melodies, a periodic noise

Neurogram

• nueral representation of Sound 

- defined by property of tonotpopy - how pitch/frewuency information is being activated in brain - how is it laid out

Periodotopy

• linear spatial arrangement of speech envelope 

  • Right hemisphere tracking speech - you know where you are at a discourse level - of speech

Receptive field of neuron

• connection between a certain stimulus and the response of a given neuron

Phonotopy

phonemes with similar fearures activate nearby nueronal populations 

Categorical perception

Sound is gradient

perception is categorical.

Auditory perception + linguistic theory

Using EEG we can see 

• Gradient acoustic encoding.

• Categorical symbolic representations.

• Decomposing features of symbolic representations.

• Dedicated areas that are respondent to and tracking speech sounds 

• Manner 

• Place - in IPA - based on where your articulators - lips - come into contact 

• Voice - is it voiced or not 

• Sound

Mismatch response (MMR)

MMR  - a signal that emerges around 200 ms after hearing a deviant item vs when its presented as a standard

MMR 

• Sensitive to phonemic difference, not just acoustic differences

• Expected since brain has rapid sensitivity - less than 200 ms for phonemic features 

• Used to understand whether different stimuli match or not 

  • If 2 ppl with different sounding voices make the same speech sound - does it activate the same or different mental representations in a listener?

Mismatch Response - Oddball design

Oddball design 

In odd ball experiment there are 

• Standards - Certain stimuli are presented frequently - 80% of time 

• Deviants - some stimuli presented rarely - 20% 

2 pieces of evidence that speech perception can’t be understood using just continuous info 

• mainly be speech signals vary greatly & variation handled with ease

Evidence 1 : Categorical perception - phenomenon stating that if you have speech sounds that have been artificially altered & start with 1 syllable, then  change incrementally to another, listeners don't perciere a smooth change Ganong effect - If hearing ambiguous sound between [s] vs [ʃ] (sh sound) they'll percive whichever one makes more sense in a word - more likely to hear [s] for a male voice : “said” -more likely to hear [ʃ] for a female voice : “shed” Evidence 2 :electrodes show classic non- linear response associated with categorical perception Specific electrode responds in an "all or nothing" way when a consonant is voiced one responds in "all or nothing" when consonant unvoiced one responds to stops regardless if voiced or not Even carry responses show categorical, not continuous features  Categories can’t fully be understood as summaries of acoustic differences

2 routes we take when doing the phonological processing of words 

Dorsal pathway - sound to articulation

Ventral pathway- - sound to meaning

Dorsal pathway 

Sound to articulation Where things are located in space  Hear something - maps it to part of brain that does motor gestures  Affects people’s perception because you map your system to motor system when you hear it  We use motor system when linking

Ventral pathway 

Ventral pathway  Sound to meaning Want to know what sound is   Map to meaning so you know what something is when you hear it

IFG - Dorsal stream

 Broca’s area is part of the dorsal steam that maps speech sounds to articulatory representations. 

How does this fit with what we know about Broca’s aphasia?

Voice onset time

• Voice onset time - Time delay between release of consonant (burst) and onset of voicing 

• Different VOT between p(h)- aspirated, p, and b 

mismatch negativity

•  (MMN) is an automatic, (ERP) component recorded via (EEG) that reflects the brain's ability to detect sensory or auditory changes

• Oddball paradigm

Zeng 2002 experiment

Study - experimenters lateralized sound properties into different ears

• phonemic feature length vs syllable length

• the study demonstrated a clear hemispheric lateralization preference: