w3 l1 - speech perception and production 1

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/33

encourage image

There's no tags or description

Looks like no tags are added yet.

34 Terms

1
New cards

how did we used to study these

  • Used to study the two separately, however they are very tightly linked

  • Idea of how they are link come from dubbing - not that common for english speaking

  • Clear mismatch between sounds we are perceiving and movement of the mouth


2
New cards

Speech perception is hard!

  • Each circle is a vowel category 

  • Peterson & Barney (1952), Fig. 8

  • Participants (speakers) said “heed”, “hid”, “head”, “had”, etc…


<ul><li><p><span style="background-color: transparent;">Each circle is a vowel category&nbsp;</span></p></li><li><p><span style="background-color: transparent;">Peterson &amp; Barney (1952), Fig. 8</span></p></li><li><p><span style="background-color: transparent;">Participants (speakers) said “heed”, “hid”, “head”, “had”, etc…</span></p></li></ul><p></p>
3
New cards

Speech production is also hard!

  • There is no one-to-one mapping between vocal tract configurations and speech output

  • Tongue and mouth have to move and coordinate in a very quick and precise manner in order for us to produce speech 

  • Not just the speed, in order to produce the same sounds - use dif configuration of muscle tongue etc in dif positions, because:

  • Speech sounds are highly influenced by the sounds surrounding them - coarticulation


4
New cards

Coarticulation

  • anticipatory

  • carryover


5
New cards

anticipatory

  • Say the word “stew”. When do you start lip-rounding?

  • Compare the place of articulation for [k] when you say “ski” compared to saying “skunk” - position of tongue slightly dif, affected by the vowel after the k sound 


6
New cards

Carryover:

  • Say the word “happen”. What kind of nasal sound do you produce at the end, and why? Bit bilabial, sounds like a m, because of the p sounds that precede it 


7
New cards

Variation from speaker, region and context

Variation in how we produce speech sounds, even in the same lang. Eg region, gender, age, where in the word the sound is produced 



8
New cards

 A computational model learning to speak 

  • Fact that is so complex is underscored by how hard it is to teach computers how to speak

  • Way these computational models speak is fundamentally dif from the way we speak, text to speech, uses sounds. Does not use vocal cords to produce speech sounds 

  • Using vocal cords in computers is a lot harder 

  • Shows just how hard it is 

  • DIVA: Directions into Velocities of Articulators

  • (Guenther & Hickok, 2016)


9
New cards

A solution to the problem?

  • A tight link between speech perception and speech production, can help us understand 

  • When we listen, we activate some of the same representations and processes that we use to speak and that we have learned as children

  • Strong version: Motor Theory of Speech Perception

  • Weaker version: Dual-route Model of speech processing,


10
New cards

Motor Theory of Speech Perception

  • early theory, when we perceive speech sounds, what our brain represents in the movement our mouth moves ??. idea implies that whenever we perceive speech, we use the areas of the brain when we speak ourselves. Activation of motor areas in the brain is needed to understand speech sounds. Strong idea, not well believed 


11
New cards

Dual-route Model of speech processing

  • still suggests motor region of  brain is active when speaking, but this does not affect comprehension


12
New cards

Dual-route model of speech processing (Hickok and Poeppel)

  • When listen to speech, two dif streams of info processing that go on at the same time. Dif areas of brain  

Components of the model:

  • Ventral stream processes speech signals for comprehension (“WHAT” pathway) meaning of what we are listening to 

  • Dorsal stream processes acoustic information and translates it to articulatory representations used in speech production (“HOW” pathway)


<ul><li><p><span style="background-color: transparent;">When listen to speech, two dif streams of info processing that go on at the same time. Dif areas of brain&nbsp;&nbsp;</span></p></li></ul><p><span style="background-color: transparent;">Components of the model:</span></p><ul><li><p><span>Ventral stream</span><span style="background-color: transparent;"> processes speech signals for comprehension (“WHAT” pathway) meaning of what we are listening to&nbsp;</span></p></li><li><p><span>Dorsal stream</span><span style="background-color: transparent;"> processes acoustic information and translates it to articulatory representations used in speech production (“HOW” pathway)</span></p></li></ul><p></p>
13
New cards

other representation

knowt flashcard image
14
New cards

Motor activation during speech perception

  • Key piece of evidence is that we observe activation of motor areas in the brain in passive speech perception 


15
New cards

Transcranial Magnetic Stimulation (TMS)

  • Use a coil to induce an electrical current in a specific area of the brain, can be excitatory or inhibitory

  • Can be incredibly targeted 


16
New cards

TMS study: Watkins et al. (2003)

  • Combined tms with recording of motor evoked potentials from muscles that control the lips 

  • So if stimulate a particular area of the brain, will you detect electrical activity of muscles which move the lips

  • Is activation of the lips greater for certain conditions 


17
New cards

different conditions

  • Speech condition - watched vid of someone speaking  

  • non-verbal - sounds but not speech 

  • Lips - watched vid of someones lip moving without audio 

  • Eyes - watched vid of the eye 

  • Control experiment: the same four conditions with TMS in motor hand area, measured at hand muscles


18
New cards

TMS study: Watkins et al. (2003) - Results

  • When applied tms to motor face area, especially left hem, see modulation of electrical activity recorded from the lips 

  • Activity a lot higher when listening to speech and seeing someone's lips moving compared to other conditions 

  • So even just seeing lip movement results in activation of motor areas of the brain



<ul><li><p><span style="background-color: transparent;">When applied tms to motor face area, especially left hem, see modulation of electrical activity recorded from the lips&nbsp;</span></p></li><li><p><span style="background-color: transparent;">Activity a lot higher when listening to speech and seeing someone's lips moving compared to other conditions&nbsp;</span></p></li><li><p><span style="background-color: transparent;">So even just seeing lip movement results in activation of motor areas of the brain</span></p></li></ul><p><br></p>
19
New cards

Why is the motor system activated while listening to speech?

Two possibilities:

  • Simulation - strong explanation, favoured by components of the motor theory of speech perception

  • Mentally simulating the movements used to produce that speech

  • An epiphenomenon - favoured by dual model of perception 

  • Activation of motor system when listen to speech is a bi-product of something else. Does not play an important role in speech perception 


20
New cards

Simulation-based perception of movements

  • Recorded pps throwing darts 

  • Played them the videos, observed themselves and other people doing the dart throwing action 

  • Predict for the videos where the dart would land on the target 

  • Prediction is more accurate for self-generated than other-generated movements

  • Even when blocked out face and had a period of time between sessions 

  • When movements are your own, better at running a simulation of your own movements 


21
New cards

A role for motor involvement in speech perception?

  • Does it actually make our perception better:

  • Does it enhance perception of individual phonemes?

  • Does it enhance higher-level comprehension processes?


22
New cards

TMS study: Schomers et al. (2014)

  • Pps listen to words

  • Which picture on the screen matches the word - focused on the meaning of the word 

  • Whilst doing this applied tms to lip or tongue area 

  • Chose words that started with a bilabial phoneme (strong involvement of lips)

  • Other word eg tool (no movement of lips when produce words, instead strong involvement of tongue)


Key question:

  • Is there a role for the left articulatory motor cortex in word comprehension?


23
New cards

results

  • Grey bars are control - performance on lip and tongue words doesn't differ a lot 

  • When tms lip - people are much faster when responding to lip words compared to tongue words. Increasing processing in lip area facilitated perception and comprehension of words with a bilabial consonant 

  • First evidence of a causal relationship between motor system and higher-level language comprehension

  • But….are all results as expected?

  • Same effect not found for tongue - not sure why


<ul><li><p><span style="background-color: transparent;">Grey bars are control - performance on lip and tongue words doesn't differ a lot&nbsp;</span></p></li><li><p><span style="background-color: transparent;">When tms lip - people are much faster when responding to lip words compared to tongue words. Increasing processing in lip area facilitated perception and comprehension of words with a bilabial consonant&nbsp;</span></p></li></ul><ul><li><p><span style="background-color: transparent;">First evidence of a causal relationship between motor system and higher-level language comprehension</span></p></li><li><p><span style="background-color: transparent;">But….are all results as expected?</span></p></li><li><p><span style="background-color: transparent;">Same effect not found for tongue - not sure why</span></p></li></ul><p></p>
24
New cards

What role does the motor system play in speech perception?

  • Motor Theory of Speech Perception: Activation of the motor system is necessary for speech perception 

  • But what is the functional role of motor activation? Why do we see this happening 

  • (Galantucci et al., 2006)


25
New cards

Possible functional roles of motor activation

  • Automatic mirroring imitation - rooted in work on mirror neurons in primates 

  • Perceptual fill-in (most relevant in noise) more pronounced and important when what we're listening to is noisy, can't just rely on auditory info

  • Recognition  (via simulation)


26
New cards

2. Motor activation during speech perception is epiphenomenal - an alternative

  • Yes they are linked, not because need production for comprehension, needed to learn to talk. Then carries on in adulthood 

  • Dual-Route Model: Activation of the motor system during speech perception is a by-product of sensory–motor integration


27
New cards

Sensory–motor integration plays a key role in

  • Learning to speak - had to learn to associate movement with sounds that came out, during this learning links to perception and production stays with us. Doesn't play any real role as adults. 

  • Monitoring and adaptation. But learning doesnt stop when babies, continues as an adult, need to adapt way of speaking and perceptual abilities to understand dif accents etc 

  • Need to rely on that link when monitoring your own speech. Even if no more adaptation to do, need to listen to output of own speech to catch errors 


28
New cards

Adaptation to accents

As experience with an accent increases, find it easier to understand

29
New cards

A role for the motor system in accent adaptation?

Are people better at understanding a novel accent if they are trained to try and imitate that accent?

30
New cards

Adaptation study: Adank et al. (2010)

  • Dutch listeners exposed to an artificial novel accent of Dutch

  • Long (lax) vowels become short (tense) and vice versa; 

  • e.g., De bal vloog over de schutting “The ball flew over the fence” pronounced as 

  • “De baal flog offer de schuuttieng”


31
New cards

conditions and method

  • Had a lot of dif conditions 

  • Some exposed to lots of sentences 

  • Some repeat

  • Transcribe using phonetic symbols 

  • Asked to imitate in 2 conditions 


32
New cards

results

Only the two imitation training conditions improve understanding more than the baseline



<p><span style="background-color: transparent;">Only the two imitation training conditions improve understanding more than the baseline</span></p><p><br></p>
33
New cards

By activating the motor system, speed up adaptation to an accent

  • Being exposed to an accent improves understanding of that accent

  • Trying to imitate the accent, thus engaging the motor system, improves understanding relative to passive exposure


34
New cards

Summary

  • Speech production and speech perception are closely linked

  • When we learn to speak, we are trying to reproduce the sounds we hear

  • When the sounds change (e.g., when we encounter a new accent), we adapt our perception

  • And eventually we may adapt our production too

  • The motor system is active when we listen to speech

  • But the exact function of this activation is debated - more researchers believe that it is just a bi-product 

  • It may reflect simulation of the sounds we hear, and that simulation may help perception