Introduction to Immersive Multimedia: AR, VR, and MR Systems

Traditional vs. Digital Media Categories

• Traditional Media: Examples include posters, printed materials, radio, and television (TV).
• Digital Media: This category encompasses Facebook ads and online applications, such as gambling apps.
• Interactive Media: These platforms allow user participation and interaction. Examples include:
  • TikTok
  • Facebook Live sessions
  • AR (Augmented Reality), VR (Virtual Reality), and PR (Physical Reality) integrations.

The Evolution of Immersive Multimedia

• Historical Progression: The media landscape has evolved from traditional formats (print, radio, TV) to contemporary AI-powered immersion.
• Spectrum of Digital Multimedia: This field includes video, audio, web content, interactive games, and the full immersive spectrum known as XR (Extended Reality), which consists of AR, VR, and AI-powered experiences.
• Definition of Immersive Multimedia: This refers to content and technology that permits users to go beyond watching or reading. It enables users to interact with and feel physically present inside a digital environment.

Augmented Reality (AR) Fundamentals

• Definition: Augmented Reality overlays digital images and text onto the user's view of the physical world. Unlike fully immersive systems, the user can still see reality through a digital screen or glass.
• Technical Mechanism: Software is used to recognize surfaces, markers, and specific images. These elements act as anchors for digital content within the physical space.
• Types of AR Implementation:
  • Smartphone AR: The most common form, utilizing mobile device cameras and screens.
  • Camera-based or Marker AR: This is triggered by scanning a specific image or object to reveal digital overlays.
  • Wearable AR: This is a fast-growing category that includes hardware such as smart glasses.
• Practical Examples and Applications:
  • Pokémon GO: A primary example of catching digital characters within the real-world environment.
  • Social Filters: Visual overlays used on social media platforms.
  • Navigation: Digital directions overlaid on real-world paths.
  • Industry Use Cases: Retail, education, and tourism.

Virtual Reality (VR) and Degrees of Freedom

• Definition: Virtual Reality fully replaces the user's visual field with a computer-generated environment. This typically requires a VR headset that covers the eyes to isolate the user from physical reality.
• Degrees of Freedom (DOF): This concept tracks how a user can move within the virtual space.
  • $3\text{-DOF}$ (Three Degrees of Freedom): This tracks head rotation only. Users can look around their environment while remaining in a seated position.
  • $6\text{-DOF}$ (Six Degrees of Freedom): This tracks both head rotation and body movement through physical space, allowing for total immersion.
• Interactivity: Interaction is often facilitated through controllers or hand tracking. This allows users to grab and manipulate virtual objects naturally within the simulation.
• Applications of VR:
  • Gaming: Fully immersive and interactive gaming and sports.
  • Professional Fields: Healthcare, architecture, and training simulations.
  • Labs and Tourism: Virtual tourism and location-based laboratories.

Mixed Reality (MR) and Extended Reality (XR)

• Mixed Reality (MR): This is an advanced subset of the immersive spectrum. It allows users to interact with digital objects while maintaining a view of their actual environment.
• Hardware Example: The Apple Vision (Apple Vision Pro) is a key example of MR. It allows the user to see their real surroundings while dragging and manipulating virtual objects as if they existed in the physical room.
• Extended Reality (XR): This is the umbrella term used to describe the entire spectrum of immersive technologies, spanning from simple AR to fully immersive AI-powered environments.