NGUI: Chapter 9

Historical Origins of Data Physicalisation

  • Mesopotamian Clay Tokens (8000 BCE): Historically, physical artifacts were utilized for data tracking long before digital systems. These clay tokens served as one of the earliest forms of recording information and managing inventory.

  • Quipu (2500 BC): An ancient record-keeping system used in the Andean region. It consisted of knotted strings where the number, type, and position of knots, as well as the color and twist of the strings, encoded information such as statistical data and census records.

Defining Data Physicalisation and Data Sculptures

  • Data Physicalisation (Physicalization): A physical artifact whose geometry or material properties encode data.

  • Data Physicalisation as a Research Area: This field explores how computer-supported physical representations of data (physicalizations) can support human cognition, communication, learning, problem solving, and decision making.

  • Terminology: The term Physical Visualisation is sometimes utilized as a synonym for data physicalisations.

  • Data Sculptures: These are specifically defined as artistic data physicalisations. A data sculpture is a data-based physical artifact that possesses both artistic and functional qualities. It aims to augment an audience's understanding of data insights and the underlying socially relevant issues.

Multisensory Data Representation

  • This approach involves displaying data in a way that engages multiple senses simultaneously, including sight, hearing, and touch.

  • Objectives: The goal is to enhance the comprehension and interpretation of complex datasets.

  • Representational Capacity: By moving beyond visual-only cues and incorporating other sensory modalities, it becomes possible to represent a larger number of variables and complex relationships within the same physical instance.

Relevant Domains and Tasks

  • Data physicalisation is grounded in several interconnected domains:

    • Information visualisation and scientific visualisation.

    • Tangible User Interfaces (TUIs).

    • Shape-changing interfaces.

    • Human-computer interaction (HCI) and design.

    • Electrical engineering.

  • Primary Tasks: Similar to digital information visualisation, physicalisations are used to:

    • Discover: Uncover patterns or insights within the data.

    • Present: Communicate data findings to others.

    • Enjoy: Provide an aesthetic or engaging experience for the user.

Key Advantages of Data Physicalisation

  • Tactile Exploration and Embodied Interaction: Physical artifacts allow for direct manipulation, utilizing the concept of embodied cognition where physical engagement leads to deeper insights.

  • Active Participation: The physical nature of the data encourages higher levels of user engagement compared to passive viewing.

  • Accessibility: Physicalisations provide an alternative for users with visual impairments by using modalities like touch (haptic feedback) to convey information.

  • Efficiency in Data Perception: Research indicates that interactions with physicalisations can be more efficient than digital counterparts. For example, in a study comparing physical 3D3D bar charts to on-screen 3D3D bar charts, the physical charts outperformed the digital ones. This is attributed to:

    • The major role of touch in interaction.

    • The use of fingers to mark specific parts of the chart, acting as external cognitive and visual aids.

Enabling Technologies for Physicalisation

  • Digital Fabrication Tools: Essential for creating static physicalisations, including 3D3D printers, laser cutters, and CNC machines.

  • Actuated Tangible Interfaces: These involve physical objects that can dynamically change their form, position, or other physical properties.

  • Specialized Output Devices:

    • Shape-changing displays.

    • Olfactory (scent-based) output devices.

    • Haptic devices such as gloves.

Prototyping and Framework Examples

  • Activity Sculptures: These are used as tools for self-reflection (the "quantified self"). For example, sculpted representations of running activities can spark social dynamics and discussions about performance.

  • EMERGE: A physical and interactive 3D3D bar chart.

  • inFORM/TRANSFORM: Technology used to create dynamic physical surfaces, often represented by the mathematical function z=r2−y2z = r^2 - y^2.

  • DataLev: A design space and physicalisation platform that utilizes acoustic levitation. It uses physical props and levitated materials combined with path planning and digital imaging to create mid-air physicalisations.

  • Sensor Bricks: A collaborative tangible sensor toolkit designed to support the development of data literacy. It allows users to select input bricks (e.g., Sonic sensors) and output bricks to build sensor systems, displaying values such as 42.742.7 or 0.020.02.

  • Braille Smartwatch and Dot Pad: Tactile displays designed for providing accessibility through touch-based graphics and text.

  • DataPhys for Geo-tagged Data: Developed by Ghassan Saliby, this approach uses multiple physical variables—such as shape change (3D3D pixels), vibration, and temperature (via Peltier modules)—to physicalise datasets like Covid-19 data for Europe.

Research Challenges in the Field

  • Transition to Dynamic Physicalisation: Most existing applications are static; there is a critical need for dynamic forms that can change as data updates.

  • Data-Driven Guidelines: A fundamental definition of data physicalisation guidelines is required to enable automated, data-driven creations.

  • Design Space Understanding: Researchers must better understand physical encoding, physical variables, and the perceptual effectiveness of combining different variables.

  • Systematic Frameworks: There is a need for conceptual software frameworks to support rapid prototyping and the evaluation of physical encoding effectiveness.

  • Interaction Design: Establishing guidelines for combining physical interaction with synthetic (digital) interactions, including new multi-user interaction models.

Dynamic Data Perceptualisation at WISE

  • The WISE research lab explores using physical objects for both input and output.

  • Physical Variables: The lab investigates the use of variables like temperature, texture, friction, and smoothness for data exploration.

  • TangHo Platform: An experimental tangible holograms platform. It features a body-mounted 6DOF6\text{DOF} (six degrees of freedom) arm prototype built with Lego Mindstorms.

    • The motors can be locked or set to "float" mode for bidirectional Input/Output.

    • It uses a replaceable sphere for non-visual feedback.

    • Challenges include inverse kinematics (solved via MATLAB Robotics Toolbox), limited HoloLens hand tracking, and the eventual 3D3D printing of the final arm design.

    • Interaction coordinates examples: (X:−0.29,Y:0.22,Z:0.22)(X: -0.29, Y: 0.22, Z: 0.22) and (X:−0.12,Y:0.32,Z:0.22)(X: -0.12, Y: 0.32, Z: 0.22).

  • Data Perceptualisation Framework:

    • PhysDIO: A device-independent output layer dealing with data points, data variables, and physical variables.

    • The framework levels include User Level (Specification, Assembly, Selection, Interaction/Feedback) and Framework Level (Processing, Output, Intermediary/PhysDIO, Physicalisations, Representations, Data Interpreter).

    • Goal: Determining "just-noticeable differences" for human sensitivity to specific physical variables.

Radical Atoms: The Future of Human-Material Interaction

  • Proposed by Hiroshi Ishii and colleagues (2012), Radical Atoms is a vision that moves beyond "Tangible Bits."

  • Concept: A hypothetical generation of computationally transformable and reconfigurable materials.

  • Key Features:

    • The material is bidirectionally coupled with an underlying digital model (bits).

    • It can transform its shape, conform to physical constraints, and inform users of its own affordances.

    • Material User Interface (MUI): A future where all digital information has a physical manifestation, allows for direct, physical interaction with data.

Multisensory Examples and Output Modalities

  • Taste the TV (TTTV): A device capable of mimicking flavors.

  • 3D Printed Food: Customizing food based on data. For example, files for "Jamie Oliver’s Filet" might encode nutritional data such as 76397639 calories and 14401440 protein (units contextually defined).

  • Olfactory Interfaces: Systems that provide scent-based feedback, with experimental data logging temperatures reaching up to 60 ∘C60 \text{ }^{\circ}\text{C} over a 40 s40\text{ s} duration.

  • Weart TouchDIVER: A haptic glove used for fine-grained tactile feedback.

  • Real-Time Cross-Media Data Exploration: Tools integrating weather data (e.g., 79%79\% Humidity, 68%68\% Rain, Wind E 10 mE\,10\,m) with physical and digital viewers.