PSY30520 Spring 2025 - Language: From Sounds to Words, Lexical and Semantic Language Processing

A study participant listens to audio clips of different scenarios (shopping mall, basketball game, wedding, etc.). Then, they listen to the clips again, along with new clips, and are asked:
- Whether they heard the clips before.
- Whether they remembered the moment or just knew it had been played before.
If the person's brain was scanned using fMRI during the second half of the task, we would expect to see:
- Increased activity in the hippocampus for audio clips that are recollected (remembered the moment). This pattern was originally seen in Eldridge et al. (2000).
- Increased activity in the perirhinal cortex for audio clips that are familiar (just knew it had been played before).
- Activity in perirhinal cortex scales with confidence in recognition (Ranganath et al. 2004).
- Some people mentioned the prefrontal cortex, but that was specifically because Patient Jon was being asked self-relevant info.

Midterm 2 viewings:
- Wednesday 4/9 1-4:30PM (Corbett 355A)
- Thursday 4/10 2-3PM (Corbett 546)
- Friday 4/11 2-3PM (Corbett 546)
Vanessa’s office hours, Monday 4/14 4:30-6PM (Hagerty)

Goal of language:
- Communicate ideas (abstract or concrete) to others.
- Requires coherent encoding/decoding of semantic information into verbalized/written/signed units for language production and understanding.
Three main components:
- Phonology -> sound
- Syntax -> grammar
- Semantics -> meaning

Rules of sounds of language: governs what sounds can appear in different parts of a word, for a given language.
Phonemes
- Smallest unit of sound that can signal meaning.
- /p/ vs /b/ → “pill” vs. “bill”
Phonetics
- How is the unit produced?
- /p/ vs /p/ → “pill” vs. “spill”

Sound is transmitted up the ascending auditory pathway in terms of the frequencies that are present in a given stimulus.
Somewhere beyond the primary auditory cortex, neurons are responding to combinations of frequencies, such as in the case of pitch.
This is also true for our understanding of speech sounds! Cano (2014), Usman (2017)

Beyond the auditory cortex is a complex network of regions (Hickok & Poeppel, 2007).
- Ventral stream: link phonological and semantic information.
- Dorsal stream: sensorimotor interface, articulation.

The superior temporal gyrus and superior temporal sulcus have both been implicated in speech processing, in different ways:
- Superior temporal gyrus: distinguishing between phonological aspects (but not necessarily speech-specific).
- Superior temporal sulcus: shows preferential responding for speech vs. non-speech sounds.
https://neurosciencenews.com/neuroscience-terms/left-mid-superior-temporal-cortex/

Recall how previously we encountered patients KF and EE, who had short-term verbal memory deficits based on lesions in temporoparietal areas and angular gyrus.
As we move out from the superior temporal gyrus, we see more speech/verbal related specificity in responses.
But these regions are not just involved in speech! (e.g., Hein & Knight, 2008).
- Mopro: motion processing
- Speechpro: speech processing
- ToM: theory of mind
- AV: audiovisual integration

You are experiencing the classic ‘cocktail party’ scenario where you’re trying to listen to one person at a party, but there are many conversations going on at the same time.
At a neural level, if you were looking for effects of attention on speech, where would you look?
- Superior temporal gyrus

Storage of representation of meanings of words in the middle temporal gyrus (MTG), while communicating with regions that are building up contextual representations (Lau et al., 2008).
- Angular gyrus
- Inferior frontal gyrus (ant. / post.)
- Anterior temporal cortex

N400: Seen with violations of semantic expectancy (does that word match?) (Kutas & Federmeier, 2000).

P600: Seen with violations of syntactic expectancy (Osterhout & Holcomb, 1992).
- “The broker persuaded to sell the stock was…”: ‘persuaded’ is transitive
- “The broker hoped to sell the stock.”: ‘hoped’ is intransitive (expect sentence to end)

Split-brain patients have shown that the RH can understand spoken and written language, but with deficits:
- Poor comprehension of complex syntax.
- Cannot produce speech output.
- Trouble processing abstract vocabulary.
RH does contribute to processing in other ways:
- Prosody (tone of speech).
- Narrative (following along with a story).
- Inference (filling in the blanks on something not explicitly said) (Meyer et al., 2002).

Processing of language involves recognition of speech components in increasingly complex ways.
Movement towards neuroimaging to understand brain networks underlying speech and language processing.
ERPs: N400, P600

Under what conditions might we see an N400? a P600?
What is an example of a sentence that would make an N400 or P600?
How has the ‘semantic P600’ illusion changed our understanding of what these ERP components might represent?