L6: Language 2: An introduction to computational models of speech perception

Develop and build on your understanding of how words are recognised from speech (Building on Level 1 Learning) and contemporary models of speech perception Understand and describe the TRACE model (Elman & McClelland, 1999) Understand and describe Revised Cohort Model (Marslen-Wilson & Warren, 1994) Evaluate the evidence that supports the TRACE model Evaluate the evidence that supports the Revised Cohort Model

bottom up processing

sensory input → semantic understanding

top down processing

semantic understanding → sensory input

challenges to lexical access

• continuous speech stream

• homonyms vs homophones

• coarticulation

• different accents

• invariance problem

coarticulation

• a challenge to lexical access

• speech production influenced by sounds that proceed and follow a phoneme, e.g. thiN book, thiN carpet

invariance problem

• problems of definition of acoustic properties- phoneme, syllable, words

• a challenge to lexical access

disambiguating the speech stream

• categorical perception

• perceptual learning

• top down processing

categorical perception

ability to distinguish between sounds on a continuum based on voice onset times (VOT)

va vs fa

→ way of disambiguating speech stream

perceptual learning

adjust categorical perception based on sounds we hear

→ way of disambiguating speech stream

what does spreading activation facilitate?

predictions of what may be coming up next via activation of items related to acoustic input

what affects speed of lexical access?

lexical characteristics

lexical characteristics affecting speed of lexical access

• word length (long words are slower to process)

• neighbourhood density (lots of neighbours - processed more slowly (fast < 10 neighbours, slow > 10 neighbours)

• frequency (more frequently a word is accessed in lexicon, the quicker u can access it)

what is lexical accessed based on?

• bottom up- acoustic input

• top down processing- disambiguating the speech stream

• lexical characteristics

• context

• spreading activation that facilitates predictions

ways lexical access could happen

1. activate words that match the sounds at each point in the unfolding speech stream

2. activate all matching words and gradually deactivate words that no longer match

3. gradually activate matching words until one word has more activation than the other words

Marslen-Wilson (1987)

The Cohort Model

Elman & McClelland (1999)

The TRACE Model

models of speech comprehension

• The Cohort Model

• The TRACE Model

Cohort Model

• predicts that we access words in the lexicon via ACTIVATION of ALL WORDS sharing INITIAL FEATURES and gradually DEACTIVATE WORDS that STOP MATCHING the features

TRACE Model

• predicts that FEATURES ACTIVATE PHONEMES that ACTIVATE WORDS with a gradual INCREASE in ACTIVATION of words that MATCH ALL FEATURES so word with MOST ACTIVATION WINS

Cohort Model Steps

• lexical activation

• lexical activation of the cohort that match the input

• gradual deactivation of items that fail to match the input

• uniqueness point = when only one word activated matches the input

• items that do not match the onset of the word are never activated

effects on Cohort Model

• neighbours compete with eachother for recognition (neighbourhood effects= words that match the acoustic input compete for activation), e.g. learning aprikol slows down recognition of word apricot

• words with high frequency have high resting states so less activation required to recognise high frequency words (frequency effects) e.g. apricot wld be recognised more quickly (ms) than low freq word aprikol

evidence to support cohort

gating experiments

(Warren & Marslen Wilson, 1987,8)

• Ps presented with fragments of words that gradually reveal whole word and asked to guess what word is after each presentation

gating paradigm: early findings

• Grosjean, 1980

• presentation of word stretcher

evidence supporting cohort: gating experiments suggest that…

• recognition of a word is a gradual process that starts from word onset and continues until end of word

• candidate words that no longer fit the acoustic input are eliminated

architecture of cohort model (Marslen-Wilson and Warren, 1994)

• facilitatory signals are sent to words that match the speech input

• inhibitory signals are sent to words that do not match the speech input

• bottom up processing has priority

bottom up or top down processing? cohort model

• gives priority to bottom up processing, which may account for the phoneme restoration effect

• sentence context does not influence the process of lexical access

• lexical selection is based on activation of phonology and semantic information

• integration is affected by sentence context (top down)

• early iterations of model suggested context constrained the cohort

Cohort model: three stages to word recognition

ascending from bottom to top (so in order of access → selection → integration):

• integration

• selection

• access

access stage

• first stage of cohort model

• acoustic-phonetic info mapped onto lexical items

selection stage

• second stage of cohort model

• candidate words that mismatch the acoustic input are deselected

• candidate word is chosen

integration stage

• third stage of cohort model

• semantic and syntactic properties of the word are integrated amd checked against the sentence

• top down processing

cohort model predictions

• items that match acoustic input but do not match sentence context are activated

• items that match acoustic input but do not match sentence context are deactivated once the word is selected

revised cohort model 1994

• context influences selection/integration of word into sentence

→ the word with semantic activation that fits the contect of the sentence will be integrated into the sentence

• the men had served for many years under their /cap/

→ semantic representation of captain is a better fit to the sentence than the semantic representation of capital and helps to single out captain as the appropriate word

how does the cohort model process context? evidence

priming paradigms

• doctor is prime word: nurse is target

• doctor and nurse are semantically related. spreading activation allows nurse to become active when doctor is presented

• sheep is prime word: nurse is target

• sheep and nurse not semantically related. presentation of sheep does not activate nurse.

lexical decision task: cross modal priming: Zwitserlood, 1989

• cross modal priming

• prime word- auditory

• target word- visual

• related prime-target pair: captain (prime, auditory); ship (target, visual)

• unrelated prime-target pair: captain (prime, auditory); wicket (target, visual)

• hear “/cap/…” → word cld be captain or capital or smth else

• target words are ship (related), money (related), wicket (unrelated)

• reaction times to respond to MONEY, SHIP, and WICKET are measured

• faster reaction times for related concepts: capTAIN- SHIP, capITAL- money, compared with control (wicket)

impact of context- biassing (same study as above)

the men around the grave mourned the loss of their cap….

• expected to find faster reaction times (and so activation) of ship, but not money or wicket

• actual findings: faster reaction times for ship and money, but not wicket.

• when whole word: the men around the grave mourned the loss of their captain , only saw faster reaction time for ship

• shows ONLY WITH WHOLE WORD DO WE SEE BIASSING EFFECT

what does Zwitserlood 1989 tell us abt processing in cohort model

• all bottom up until whole word selected

• sentence context does not influence semantic activation until whole word heard

• so suggests items matching acoustic input all activated, and items semantically related to these are activated

• as get more of word, deactivation until only one word match

• once word selected, then integration of sentence context, meaning semantic activation for selected word active but not other words

• so only see biassing effect for full word

• shows bottom up until word selected, then top down influence of context, otherwise context not relevant if part word

revised cohort model: summary

• speech perception is based on matching acoustic input to stored representations of words in the lexicon

• words are recognised via a competitive process that activates a word cohort

• cohort candidates do not actively engage with each other

• words identified when reach their uniqueness point

• cohort candidates that do not match acoustic input are eliminated

• context does not constrain activation of initial cohorts but allows for rapid elimination of candidates that do not match sentence context

TRACE Model (McClelland & Elman, 1986): quote from Joanisse & McClelland (2015), saying: in TRACE, words are recognised…

“incrementally by slowing ramping up the activation of the correct units at the phoneme and word levels”

TRACE Model steps

• gradual activation of items that matches input

• more input, gradually more activation of items matching input (e.g. hear APRIC and tape is activated but less so than april which is less so than apricot)

• full input → full word matching input most is most activated

• lexical competitive inhibition

• e.g. apricot inhibits tape, apple, apart etc, but not april, at the APRI stage

• more input → more activation of target word → that inhibits more lexical competitors

what is the TRACE model?

• implemented computational model based on connectionist principles

• processing units (nodes) correspond to mental representations of features (voicing, manner of production); phonemes; and words

TRACE Model: processing and connections

features (acoustic- phonetic patterns) → phonemes → lexical items (words)

^ bottom up processing

• each level is connected via facilitatory connections

• activation spreads up from features to lexical items

features (acoustic- phonetic patterns) ← phonemes ← lexical items (words)

^ top down processing

• facilitatory connections between levels also travel down from the lexical level to the phoneme level and the feature level

• connections between nodes within each level are inhibitory

describe the stages of the TRACE model, including example for word van

1. features activate relevant phonemes (e.g. feature information telling us the first sound is voice would activate phonemes that correspond to that feature and are voiced like /v/ and /b/ and /d/)

2. activated phoneme inhibits competitors (e.g. selection of /v/ would send inhibitory signal to other phonemes /b/ and /d/)

3. activated phonemes activate words (e.g. remaining phonemes in word van activated → so /v/ /ae/ /n/ , and these in turn activate words that matched input such as ban, cat, van, vat, vamp, etc)

4. activated word inhibits competitiors (e.g. increased activation to van as matching item wld send inhibitory signals to other words (ban, vat, etc) that had become activated)

5. top down processing increases activation of phonemes and features (e.g. top down processing (van → /v/ /ae/ /n/ → voiced) wld reinforce activation of nodes selected in previous levels)

so van is most and only activated and therefore final selection

radical activation model

• TRACE model

• Jusczyk & Luce (2002):
  

“ any consistency between input and representation may result in some degree of activation”

TRACE Model (McClelland & Elman, 1986)

• nodes influence each other according to their activation levels and strengths of connections

• activation develops as a pattern of excitation from facilitation and inhibition

• candidate words are activated based on the pattern of activation

• bottom up and top down processes

• bottom up- activation from feature to word level

• top down- activation from word to feature level

evidence for TRACE model- evidence from .. suggests…

• activation of words in lexicon

• evidence from Allopenna et al., (1998) and others suggests that words that rhyme with sounds in any parts of a word may become activated

• initial cohort of words activated in response to the speech stream is NOT limited to words with the same onset

Allopenna et al., (1998): demonstrated

• evidence for trace model, activation of words in lexicon

• using an eye tracking study, demonstrated that words with overlapping phonology that do not start with the same onset as the speech input (rhyme competitors) are activated in speech perception

Allopenna et al., (1998): study

visual world paradigm

• Ps presented with a grid that contains images of items such as a grid with a beaker, a beetle, a speaker, and a pram, and shapes like a triangle, a circle, a diamond, and a rectangle

• Ps asked to “click on the beaker and place it under the triangle”

• Ps eye movements monitored whilst complete task

• if words related to beaker are active in lexicon Ps will look towards those items

Allopenna et al., (1998): results

• when asked to move beaker under triangle, Ps looked at the beaker, the beetle, and the speaker, but not the pram

• referent - beaker

• cohort- beetle

• rhyme- speaker

• unrelated- carriage (diff trial)

• Ps looked at the Beaker and the Beetle in the first 400ms after the word was heard

• Ps also looked at Speaker between 400-600ms after word was heard

top down processing in TRACE model: what shld happen

• facilitatory links between words and phonemes shld result in more accurate detection of phonemes in words compared to non words

• Ps asked to detect a /t/ or /k/ in words (e.g. heighten) and non words (e.g. vinted) shld find it easier to identify the /t/ in heighten compared to vinten

evidence for top down processing in TRACE model

Mirman et al., (2008)

• faster identifcation of /t/ and /k/ in words than non words

• demonstrates effect of top down processing

top down processing: research questioning the superiority of top down effects

• Ps were able to accurately detect phonemes in non-words that were word like (Fraudenfelder et al., 1990): e.g. /t/ in vocabutary

• Ps failed to complete ambiguous phonemes with a phoneme that would create a word unless stimuli were degraded (McQueen, 1991): e.g. identifying sh as the final phoneme for fiss

TRACE model vs Cohort Model

• TRACE emphasises top down processing; Cohort minimises impact of top down processing

• Cohort predicts lexical access is biased towards activation of words with shared onsets; TRACE accommodates activation of rhyming competitors

• TRACE does not provide account of how context might affect speech perception and evidence also suggests there is a tendency to activate words that start with the same sounds (“ word onsets strongly determine the activation of competitors in memory “ - Jusczyk & Luce, 2002)

agreement between models of speech perception

• agree we access words in lexicon via activation of lexical representations

• agreement that activation is based on processes that involve facilitatory signals and competition

→ models take different routes to comprehension

difference between cohort model and revised cohort model

both say: activation of cohort (activation) → selection → integration

cohort says deactivation of non-candidate words