Visual Speech Prosody

prosody

music, melody, and rhythm of speech

disambiguates sentence intent

terminal rise is question terminal drop is statement

suprasegmental acoustic cues

• intonation

• duration

• loudness

across multiple segments

Visual prosody

visual movements of lips, head, face, body to represent speech prosody

correlated with suprasegmental acoustics

can help people determine sentence intent

Audio Visual speech

speaking: acoustic and visual cues

gesticulation:

• visual speech cues connected to speech (mouth movement)

• co-speech gestures accompany speech (waving)

Lombard effect

predicts speakers to adapt speech in response to noise

loudness, duration, spectral modulation (spectral tilt increase energy in higher frequencies )