SiR - Lecture 3

What is NVC?

Communication without words (gaze, gestures, tone, movement, lights).

Why is NVC important?

Most human communication is non-verbal, not speech → robots must use it too.

NVC modalities (list)

Gaze, gestures, expressive motion, proxemics, haptics, prosody, robot-specific cues (lights, sound).

Embodied interaction

Communication that happens in real time and physical space.

Two sides of robot communication

• Robot → human: express states clearly.

• Human → robot: robot interprets social signals.

What is glaze?

Where and how someone looks; extremely important for social meaning

Mutual gaze

Direct eye contact → attention.

Deictic gaze

“Pointing” with eyes to an object.

Joint attention

Human + robot look at the same thing together.

3 Types of eye movements

• Fixation – steady focus

• Saccades – quick jumps

• Smooth pursuit – continuous tracking (e.g., follow a moving object)

Iconic gesture (definition)

A gesture that visually represents something real — like the shape, size, or movement of an object or action (Example: tracing a circle in the air to show a ball.)

Metaphoric gesture (definition)

A gesture that represents an abstract idea using a symbolic movement. (Example: spreading hands apart to show “a big idea” or “expanding.”)

Deictic gesture (definition)

Pointing to a location or object.

Beat gesture (definition)

Small rhythmic movements matching speech.

Gesture recognition methods

RGB camera, depth camera (Kinect), motion capture, gloves (Pison).

Model Gesticulator (2020)

AI model generating gestures automatically from speech.

Expressive motion (definition)

Motion that is functional (serves a task) but also expressive (shows meaning or emotion). Example: a robot moving smoothly vs. abruptly to show confidence or hesitation.

Legible motion (definition)

Movement that humans can easily interpret (Dragan 2015).

Laban-inspired motion (definition)

Dance-based movement for mobile robots (Knight et al., 2014).

Proxemics (definition)

Study of spatial distances in social interactions

4 distance zones

Intimate / Personal / Social / Public

What affects personal space?

Personal factors (age, gender etc.), environment, culture.

Haptics (definition)

Communication through touch, pressure, force

Example: PARO robot (hint: touch)

Therapeutic robot responding to touch.

What project is the Baxter contact improvisation?

Baxter contact improvisation is a project where a Baxter robot and a human dance together, using shared weight, touch, and movement to communicate.

What is prosody?

Tone, rhythm, stress, intonation of speech → how something is said.

Why is prosody important for robots?

Shows emotion + helps detect user states.

Robot-specific NVC (mention some examples)

Lights, sounds, robot-unique motions.

Multimodal NVC

Using multiple cues at once (gaze + gesture + sound).

Challenges of multimodal NVC

Hard to synchronize; too many signals can confuse people.

Why multimodal?

More natural, engaging, trustworthy, expressive.

Why is NVC essential for social robots?

It creates transparency, trust, fluid interaction, and human-like social presence.