SiR - Lecture 4

What is Dialogue Management (DM)?

System that tracks the conversation and decides the robot’s next action.

DM Pipeline (order)

ASR → NLU → DM → NLG → TTS
Speech → Text → Decide → Generate → Speak.

Why is DM difficult? (list)

• Situated grounding

• Confirmations

• Clarifications

• Repairs

• Repetitions

• Common sense

Situated grounding (definition)

Interpreting language in a real physical context. Example: “Grab the blue cup next to the red block.”

Repair (definition)

Fixing misunderstandings.
Example: “We meet at 6—no, 7:30.”

Four functions of DM (Traum & Larsson, 2003)

• Update context

• Provide expectations for interpretation

• Coordinate with other modules

• Decide what to say/do next

Handcrafted DM (definition)

Human-designed rules or structures (no learning).
Pros: control.
Cons: rigid.

Rule-Based Approach

Uses if–then rules, no context memory.
Simple but inflexible.

Finite-State Machines (FSMs)

Fixed states + transitions.
Good for simple tasks.
Bad at handling topic shifts.

Frame-Based Approach

Uses slots (values) instead of fixed structure.
Can fill multiple slots at once.
More flexible than FSM.

Model-Based Approach

Adds user model + context model.
Robot uses past info to choose next action.
More adaptive but complex.

Statistical (ML) Approaches

DM learned from data, not rules.
More adaptable.

Corpus Creation (definition)

Collecting human–human or human–robot dialogue data to train ML models.

Example-Based Approach (definition)

Finds similar past dialogues and copies best answer.
Uses cosine similarity.

Cosine Similarity (definition)

Measures how similar two text vectors are.

• “hello” → [0.2, 0.1, 0.7]

• “hi” → [0.21, 0.09, 0.69]

MDP-Based DM (definition)

Uses Markov Decision Processes (S, A, T, R) + Reinforcement Learning. Learns optimal responses from rewards.

POMDP (definition)

Partially Observable MDP — used when the robot cannot see the full conversation state (e.g., uncertain user intent).

End-to-End Neural DM (definition)

Neural networks that generate responses word-by-word (e.g., LLMs).
Pros: flexible.
Cons: no control, needs lots of data.

Hybrid DM Approach (definition)

Combines handcrafted structure + ML learning.
Balance between control and adaptability.

When to use handcrafted DM?

When you need control, safety, or have little data.

When to use ML-based DM?

When you have a lot of data and need flexibility.

When to use hybrid DM?

When you want control + some learning, or have medium data.

Evaluation — Subjective metrics

User satisfaction, naturalness, trust, Godspeed, SSI.

Evaluation — Objective metrics

Task success rate, # dialogue turns, completion time.

Conversational Analysis — Turn-taking

Managing who speaks when.

Adjacency Pairs (definition & examples)

Paired social actions: greeting–greeting, question–answer, offer–accept/deny.

Sequence Organization (definition)

How conversations are structured logically.

Repair (in CA) (definition)

Fixing breakdowns in communication.

Common Ground (definition)

Shared understanding built up during conversation.

Conversational UX Design (definition)

Using conversational patterns to design natural, reusable dialogues.

IECR Framework (Intent–Entity–Context Recognition)

Intent = what user wants
Entity = details
Context = situation information

Example: 

“I want vegetarian pasta with onions.”

Intent = AddPreferences
Entities = vegetarian, onions
Context = meal planning

Novelty Effect (definition)

People like robots more at first simply because they’re new.

Robots in the Wild (definition)

Testing robots in real-life settings to see true user behavior when novelty fades.

Capability Communication (definition)

Robot explaining what it can and cannot do to reduce user uncertainty.

Why capability communication matters

Builds trust, reduces confusion, aligns expectations.

Communication styles (list 3)

• Baseline: only repeats when confused

• Reactive: explains when issues occur

• Proactive: introduces capabilities before user needs them