Lecture 6: Computational models of speech perception

accessing the mental lexicon

-mental representation of what we know

-all of these become active at once when perceiving what is being spoken to you:

1. phonological representation activated

2. syntax

3. semantic representation

4. orthographic representation

challenges to lexical access

-continuous speech stream

-homonyms and homophones

-co-articulation 

-different accents 

-invariance problem 

segmentation (challenges to lexical access) 

-separating out or distinguishing phonemes 

-words form the pattern of speech sounds 

co-articulation (challenges to lexical access) 

-speaker’s pronunciation of a phoneme depends on the preceding and following phonemes

-increases the variability of the speech stream

categorical perception (disambiguating the speech stream)

-ability to distinguish between sounds on a continuum based on voice onset time

perceptual learning (disambiguating the speech stream)

-adjust categorical perception based on sounds we hear

top-down processing (disambiguating the speech stream)

-if we only hear part of a word/sentence we fill in the gaps using our knowledge of the words that we did hear

-phonemic restoration effect 

spreading activation

-facilitates predictions of what may be coming up next via activation of items related to the acoustic input

-top-down knowledge allows us to ascribe meaning and predict what will be said

-activate words that sound similar to word

-if speech is degraded we can select what is most appropriate from a particular cohort

lexical characteristics

-affect speed of lexical access

• word length → long words are slower to process

• frequency → more frequently word is accessed in lexicon, the quicker you can access it

• neighbourhood density → words with lots of neighbours are processed more slowly

theories of matching sounds to mental representations

1. activate words that match the sounds at each point in the unfolding speech stream

2. activate all matching words and gradually deactivate words that no longer match

3. gradually activate matching words until one word has more activation than other words

models of speech comprehension

• cohort model 

• TRACE model 

cohort model (Marslen-Wilson)

-predicts we access words in the lexicon via activation of all words sharing initial features and gradually de-activate words that stop matching the features

-dominated by bottom-up acoustic input

-word-initial cohort may contain words having similar initial phonemes to the presented word rather than consisting only of words having the same initial phonemes

-context-inappropriate words are eliminated later in processing

lexical activation (cohort model)

-lexical activation of the cohort that matches the input

-early in auditory presentation of a word, all words conforming to the sound sequence heard so far become active → initial word cohort 

-there is competition among these words to be selected 

-words within the cohort are eliminated if they cease to match further information from the presented word or because they are inconsistent with the semantic or other context 

-processing continues until information from the word itself and contextual information permit elimination of all but one of the cohort words 

uniqueness point (cohort model)

-the point at which only one word is consistent with the acoustic signal

-know no other words that match that sound

neighbourhood effects (cohort model)

-items that do not match the onset of the word are not activated → even if they share the same sounds but not at the beginning of the word

-neighbours compete with each other for recognition

frequency effects (cohort model)

-words with high frequency have high resting states

-less activation required to recognise high frequency words

stages of cohort model 

1. access stage → word cohort is activated, acoustic-phonetic information is mapped onto lexical items

2. selection stage → candidate words that don’t match the acoustic input are deselected and one candidate word is chosen from the cohort

3. integration stage → word’s semantic and syntactic properties are integrated and checked against the sentence 

gating experiment - method (evidence for cohort model)

-participants presented with fragments of words that gradually reveal the whole word → this reduces the size of the cohort as only a certain number of words can match

-asked to guess what the word is after each presentation

gating experiment - result (evidence for cohort model)

-findings suggest that recognition of a word is a gradual process that starts from word onset and continues until the end of the word

-candidate words that no longer fit the acoustic input are eliminated

architecture (cohort model)

-facilitatory signals are sent to words that match the speech input

-inhibitory signals are sent to words that do not match the speech input

-bottom-up processing has priority

context (cohort model)

-sentence context does not influence the process of lexical access

-lexical selection is based on activation of phonology and semantic information

-integration is affected by sentence context

-context does not constrain activation of initial cohorts but allows for rapid elimination of candidates that do not match sentence context 

Zwitserlood - method (cross-modal priming and cohort model)

-lexical decision task after hearing part of a word → have to decide if target word is a word or non-word

-prime word given auditory

-target word given visually

Zwitserlood - results (cross-modal priming and cohort model)

-when only “cap__” was presented it was consistent with captain and capital

-lexical decisions were faster when the presented word was related in meaning to either word e.g., ship and money

-in other condition, the part word was preceded by a biasing context “men stood around the grave and mourned the loss of their cap__”

-context did not prevent activation of word capital even though it was inconsistent with the context

• context having minimal effect was predicted by the model as dominated by bottom-up processing

• context does not constrain the cohort until we have the full word

Zwitserlood - conclusions (cross-modal priming and cohort model)

-items that match acoustic input but do not match sentence context are activated

-items that match acoustic input but do not match sentence context are deactivated once the word is selected

revised cohort model (based on Zwitserlood findings)

-context influences selection/integration of word into sentence

-word with semantic activation that fits the context of the sentence will be integrated into the sentence

-semantic representation of captain is a better fit to the sentence than the semantic representation of capital and this helps to single out captain as the appropriate word

TRACE model (Elman & McClelland)

-predicts that features activate phonemes that activate words with a gradual increase in activation of words that match all features so that the word with the most activation wins

-bottom-up and top-down processes interact flexibly in spoken word recognition

TRACE model summary

-nodes influence each other according to their activation levels and strengths of connection 

-activation develops as a pattern of excitation from facilitation and inhibition

-candidate words are activated based on pattern of activation

• bottom up → activation from feature to word level

• top down → activation from word to feature level

connections between nodes (TRACE model)

-nodes correspond to mental representations of: features, phonemes and words

-feature nodes are connected to phoneme nodes and phoneme nodes connected to word nodes

-connections between levels operate in both directions and are always facilitatory → activation spreads up from features to lexical items and spreads down from lexical level to phoneme level and feature level 

-connections among units or nodes at the same level are inhibitory

-nodes influence each other in proportion to their activation levels and strengths of their interconnections

gradual activation of items that matches input (TRACE model)

-receive initial word sound and activate elements of relevant words in mental lexicon

-activate phoneme regardless of where it occurs within words

-don’t deactivate words that no longer match

-words that match input receive a heightened stage of activation

-all activated words are involved in a competitive process in which these words inhibit each other

-word with the strongest activation wins the competition

lexical competitive inhibition (TRACE model)

-once uniqueness point is reach will inhibit words that don’t have same level of activation

-words are still activated but do not receive any additional activation

-this prevents words that don’t match the input from receiving more activation → so they will not be selected in error

radical activation model (TRACE model)

-any consistency between input and representation may result in some degree of activation

Allopena - method (evidence for TRACE model)

-eye-tracking study

-participants hear the word beaker

-visual display in front of them showing a beaker, beetle, speaker and carriage

Allopena - results (evidence for TRACE model)

-

Allopena - conclusions (evidence for TRACE model)

-demonstrates that words with overlapping phonology that do not start with same onset as the speech input are activated in speech perception

-rhyming competitor was activated too

-supports TRACE model over cohort model as rhyming competitor would not be activated in cohort model as do not start with the same sounds as target word

-the initial cohort of words activated in response to the speech stream is not limited to words with the same onset → challenges cohort model

Mirman - method (evidence for TRACE model - top down processing)

-facilitatory links between words and phonemes should result in more accurate detection of phonemes in words compared to non-words

-participants asked to detect a /t/ or /k/ in words and non-words

-should find it easier to identify the sounds in words than non-words

Mirman - results (evidence for TRACE model - top down processing)

-faster identification of /t/ and /k/ in words

-demonstrates effect of top-down processing

evidence against top-down effects

-participants could accurately detect phonemes in non-words that were word like

-participants failed to complete ambiguous phonemes with a phoneme that would create a word unless stimuli were degraded

TRACE model vs cohort model

-TRACE emphasises top-down processing → cohort model minimises it 

-cohort model predicts lexical access is biased towards activation of words with shared onsets → TRACE model accommodates the activation of rhyming competitors 

-TRACE does not provide an account of how context might affect speech production 

-evidence suggests there is a tendency to activate words that start with the same sounds