Lecture 10: Immersive Communication

Immersion

• Perception of being physically present in a non-physical world

• Perception is made by surrounding users in content (images, sound, other stimuli) that provide metaphors of a similar environment

Types of Immersion

Sensory-Motoric Immersion, Cognitive, Emotional, Spatial Immersion

Sensory-Motoric Immersion

Haptic gloves, sensory feedback suits, console controllers, realistic audio/sounds

Physically sensory information, get a response to feel part of the virtual world

Cognitive Immersion

Feeling of another world is made through the content: problem-solving, freedom of choice, right level of complexity, avoiding (analysis paralysis - , downtime)

Emotional Immersion

Expresive avatars, narrative, affect-driven response

Spatial Immersion

Realistic environments, realistic interaction with objects, 360 view through AR/VR glasses, new LED dome screens

Conversational User Interfaces

a user interface for computers that emulates a conversation with a real human

the interaction with the system should have the characteristics of human conversation

intention: have them as companions

First Versions

rule-based versions - used Markov models (probabilistic model and you determine the outcome using the observed probabilities), using letters it becomes a language model

Parry

pattern matching, predefined responses that simulate through patterns typica

Alice

built using AIML, pattern matching, capable of responding to a wide range of topics and queries, making it adaptable for various applications

ANN

matching + generating something

Command-and-Control Interfaces

engaged by the users using a wake word or a button, gives the impression that it is listening + understanding, only capable of executing mapped functions (e.g. ANN

Transformers + LLMs

whisper architecture - speech model and can do transcription and translation, tries to generate a response word-by-word

Conversational Interfaces

effective conversational interfaces have a clear intent and create a sense of engagement and personalization, allowing users to interact intuitively with technology as if they were conversing with a real person, may utilize chatbots, virtual assistants, or voice-activated systems to enhance user experience, streamline tasks, and provide assistance in various applications

Gricean Maxims

Paul Grice suggested that conversations follow the cooperative principle + 4 key maxims:

quality (truthful + accurate communication), quantity (just the right amount of info), relevance (appropriate + relevant info), manner (clear, cooperative communication)

Maxim of Quality

should only convey information they believe to be true and avoid saying anything they think is false or for which they lack adequate evidence, requires honesty in communication and the responsibility to ensure that statements made are backed by facts or reliable sources, which enhances the trust in conversation and the relationship between communicators

Maxim of Quantity

be informative, make your contribution as informative as if required, don’t make your contribution more informative than is required

Maxim of Relevance

be relevant

Maxim of Manner

be clear, avoid obscurity of expression, avoid ambiguity (avoid language that can be interpreted in many ways), be brief (don’t be wordy), be orderly (provide info in order that makes sense)

Nielsen’s Heuristics Visibility of System Status

visibility of system status: CUI should always be clear whose turn in the conversation it is, should indicate clearly when it is processing, should acknowledge that it understood the intention of the user, should clearly log who said what and when

Nielsen’s Heuristics: Match between system and real world

match between system and real world: CUI’s utterances and actions should always be truthful, should set the right expectations about its actual abilities, use terminology appropriate to the user

Nielsen’s Heuristics: User control and freedom

users should be able too tart, interrupt and end the conversation, should be able to steer the conversation topic and its focus, should be able to access, change and delete any previous preferences

Nielsen’s Heuristics: Recognition rather than recall

recognition rather than recall: the timing and rhythm of turn-taking should feel natural and engaging. CUI should offer examples of the type of input it requires, should offer a list of options/examples for user decisions, should offer a summary of the agreement before the user makes their decision

Nielsen’s Heuristics: Flexibility and efficiency of use

flexibility and efficiency of use: CUI should allow use of shortcuts for advanced users, should allow autocomplete and autocorrect to speed up entry and reduce errors, should understand synonyms and variations within its domain, should understand various styles of language

Nielsen’s Heuristics: Aesthetic and minimalist design

aesthetic and minimalist design: utterance length of CUI should be appropriate, utterances of CUI should be relevant to the context, should communicate in an appropriate, pleasant and polite manner

Nielsen’s Heuristics: Error prevention

error prevention: CUI should handle common typos, misspellings, and grammatical errors, should be able to establish consensus in case of unclear input, should always ask for confirmation before executing possibly irreversible actions

Nielsen’s Heuristics: Help Users Recognise, Diagnose, and Recover from Errors

CUI should express what type of clarification is needed from the user, should allow its users to correct misunderstandings, should allow user to easily reverse or stop the execution

Nielsen’s Heuristics: Help and documentation

help and documentation: should be able to provide appropriate amount of information about itself, CUI should allow seamless continuation of the conversation after clarifications

Nielsen’s Heuristics: Consistency and Standards

utterances of CUI should be grammatically correct and consistent, should understand and respond to user’s style of language CUI, should follow conventions, guidelines and best practices of its environment

Augmented Reality

an interactive experience that combines real world with virtual 3D content
AR overlays digital elements onto
the real-world, overlaid information can be
constructive or destructive, creates the illusion of computer-generated content to be perceived
as part of the real world

Mixed Reality

interactive experience that merges real world with virtual 3D content

MR allows digital elements to interact with the real world, sometimes called internality, creates sensation that the real world is enhanced with virtual content

Virtual Reality

interactive experience that creates 3D world simulating the real world

allows immersion in completely computer-generated environments, also called simulated reality, creates the simulation that the user is in a world of its own

General Design considerations

1) make interactions follow realistic physics

2) make gestures and user poses must be realistic 3) provide appropriate + comfortable area of interaction

4) avoid occlusion