1/33
Looks like no tags are added yet.
how did we used to study these
Used to study the two separately, however they are very tightly linked
Idea of how they are link come from dubbing - not that common for english speaking
Clear mismatch between sounds we are perceiving and movement of the mouth
Speech perception is hard!
Each circle is a vowel category
Peterson & Barney (1952), Fig. 8
Participants (speakers) said “heed”, “hid”, “head”, “had”, etc…

Speech production is also hard!
There is no one-to-one mapping between vocal tract configurations and speech output
Tongue and mouth have to move and coordinate in a very quick and precise manner in order for us to produce speech
Not just the speed, in order to produce the same sounds - use dif configuration of muscle tongue etc in dif positions, because:
Speech sounds are highly influenced by the sounds surrounding them - coarticulation
Coarticulation
anticipatory
carryover
anticipatory
Say the word “stew”. When do you start lip-rounding?
Compare the place of articulation for [k] when you say “ski” compared to saying “skunk” - position of tongue slightly dif, affected by the vowel after the k sound
Carryover:
Say the word “happen”. What kind of nasal sound do you produce at the end, and why? Bit bilabial, sounds like a m, because of the p sounds that precede it
Variation from speaker, region and context
Variation in how we produce speech sounds, even in the same lang. Eg region, gender, age, where in the word the sound is produced
A computational model learning to speak
Fact that is so complex is underscored by how hard it is to teach computers how to speak
Way these computational models speak is fundamentally dif from the way we speak, text to speech, uses sounds. Does not use vocal cords to produce speech sounds
Using vocal cords in computers is a lot harder
Shows just how hard it is
DIVA: Directions into Velocities of Articulators
(Guenther & Hickok, 2016)
A solution to the problem?
A tight link between speech perception and speech production, can help us understand
When we listen, we activate some of the same representations and processes that we use to speak and that we have learned as children
Strong version: Motor Theory of Speech Perception
Weaker version: Dual-route Model of speech processing,
Motor Theory of Speech Perception
early theory, when we perceive speech sounds, what our brain represents in the movement our mouth moves ??. idea implies that whenever we perceive speech, we use the areas of the brain when we speak ourselves. Activation of motor areas in the brain is needed to understand speech sounds. Strong idea, not well believed
Dual-route Model of speech processing
still suggests motor region of brain is active when speaking, but this does not affect comprehension
Dual-route model of speech processing (Hickok and Poeppel)
When listen to speech, two dif streams of info processing that go on at the same time. Dif areas of brain
Components of the model:
Ventral stream processes speech signals for comprehension (“WHAT” pathway) meaning of what we are listening to
Dorsal stream processes acoustic information and translates it to articulatory representations used in speech production (“HOW” pathway)

other representation

Motor activation during speech perception
Key piece of evidence is that we observe activation of motor areas in the brain in passive speech perception
Transcranial Magnetic Stimulation (TMS)
Use a coil to induce an electrical current in a specific area of the brain, can be excitatory or inhibitory
Can be incredibly targeted
TMS study: Watkins et al. (2003)
Combined tms with recording of motor evoked potentials from muscles that control the lips
So if stimulate a particular area of the brain, will you detect electrical activity of muscles which move the lips
Is activation of the lips greater for certain conditions
different conditions
Speech condition - watched vid of someone speaking
non-verbal - sounds but not speech
Lips - watched vid of someones lip moving without audio
Eyes - watched vid of the eye
Control experiment: the same four conditions with TMS in motor hand area, measured at hand muscles
TMS study: Watkins et al. (2003) - Results
When applied tms to motor face area, especially left hem, see modulation of electrical activity recorded from the lips
Activity a lot higher when listening to speech and seeing someone's lips moving compared to other conditions
So even just seeing lip movement results in activation of motor areas of the brain

Why is the motor system activated while listening to speech?
Two possibilities:
Simulation - strong explanation, favoured by components of the motor theory of speech perception
Mentally simulating the movements used to produce that speech
An epiphenomenon - favoured by dual model of perception
Activation of motor system when listen to speech is a bi-product of something else. Does not play an important role in speech perception
Simulation-based perception of movements
Recorded pps throwing darts
Played them the videos, observed themselves and other people doing the dart throwing action
Predict for the videos where the dart would land on the target
Prediction is more accurate for self-generated than other-generated movements
Even when blocked out face and had a period of time between sessions
When movements are your own, better at running a simulation of your own movements
A role for motor involvement in speech perception?
Does it actually make our perception better:
Does it enhance perception of individual phonemes?
Does it enhance higher-level comprehension processes?
TMS study: Schomers et al. (2014)
Pps listen to words
Which picture on the screen matches the word - focused on the meaning of the word
Whilst doing this applied tms to lip or tongue area
Chose words that started with a bilabial phoneme (strong involvement of lips)
Other word eg tool (no movement of lips when produce words, instead strong involvement of tongue)
Key question:
Is there a role for the left articulatory motor cortex in word comprehension?
results
Grey bars are control - performance on lip and tongue words doesn't differ a lot
When tms lip - people are much faster when responding to lip words compared to tongue words. Increasing processing in lip area facilitated perception and comprehension of words with a bilabial consonant
First evidence of a causal relationship between motor system and higher-level language comprehension
But….are all results as expected?
Same effect not found for tongue - not sure why

What role does the motor system play in speech perception?
Motor Theory of Speech Perception: Activation of the motor system is necessary for speech perception
But what is the functional role of motor activation? Why do we see this happening
(Galantucci et al., 2006)
Possible functional roles of motor activation
Automatic mirroring imitation - rooted in work on mirror neurons in primates
Perceptual fill-in (most relevant in noise) more pronounced and important when what we're listening to is noisy, can't just rely on auditory info
Recognition (via simulation)
2. Motor activation during speech perception is epiphenomenal - an alternative
Yes they are linked, not because need production for comprehension, needed to learn to talk. Then carries on in adulthood
Dual-Route Model: Activation of the motor system during speech perception is a by-product of sensory–motor integration
Sensory–motor integration plays a key role in
Learning to speak - had to learn to associate movement with sounds that came out, during this learning links to perception and production stays with us. Doesn't play any real role as adults.
Monitoring and adaptation. But learning doesnt stop when babies, continues as an adult, need to adapt way of speaking and perceptual abilities to understand dif accents etc
Need to rely on that link when monitoring your own speech. Even if no more adaptation to do, need to listen to output of own speech to catch errors
Adaptation to accents
As experience with an accent increases, find it easier to understand
A role for the motor system in accent adaptation?
Are people better at understanding a novel accent if they are trained to try and imitate that accent?
Adaptation study: Adank et al. (2010)
Dutch listeners exposed to an artificial novel accent of Dutch
Long (lax) vowels become short (tense) and vice versa;
e.g., De bal vloog over de schutting “The ball flew over the fence” pronounced as
“De baal flog offer de schuuttieng”
conditions and method
Had a lot of dif conditions
Some exposed to lots of sentences
Some repeat
Transcribe using phonetic symbols
Asked to imitate in 2 conditions
results
Only the two imitation training conditions improve understanding more than the baseline

By activating the motor system, speed up adaptation to an accent
Being exposed to an accent improves understanding of that accent
Trying to imitate the accent, thus engaging the motor system, improves understanding relative to passive exposure
Summary
Speech production and speech perception are closely linked
When we learn to speak, we are trying to reproduce the sounds we hear
When the sounds change (e.g., when we encounter a new accent), we adapt our perception
And eventually we may adapt our production too
The motor system is active when we listen to speech
But the exact function of this activation is debated - more researchers believe that it is just a bi-product
It may reflect simulation of the sounds we hear, and that simulation may help perception