Nicola+Twilley+2017+Sight+Unseen+-+Sensory-substitution+devices+-+The+New+Yorker+May+15

Introduction to Sensory Substitution Technology

  • Article by Nicola Twilley titled "Sight Unseen."

  • The premise: "You don't see with the eyes. You see with the brain."

Case Study: Erik Weihenmayer

  • Erik Weihenmayer, a climber who lost his vision due to retinoschisis, adapts to climbing using sensory-substitution technology.

  • Uses climbing techniques such as:

    • Patting the rock face

    • Tapping with an ice axe

    • Following auditory cues from a guide wearing a bell

  • Notable achievement: First blind climber to summit Mount Everest.

The BrainPort Device

  • Description: A sensory-substitution device that allows Weihenmayer to "see" using his tongue.

  • Components:

    • A camera mounted on a band around his head

    • Electrode lollipop that sends tactile signals to his tongue (400 pixels grid)

  • Functionality:

    • Dark pixels provide strong shocks; lighter pixels provide a mild tingling sensation.

    • Described by Weihenmayer as "pictures painted with tiny bubbles."

Climbing Experience with BrainPort

  • Use of BrainPort affects climbing differently than traditional methods:

    • Slower and more deliberate movements versus fast scrambling.

    • Requires keen focus on the tactile feedback from the device.

  • Functionality: Helps restore lost hand-eye coordination; provides a better spatial understanding of climbing holds.

Growth of Sensory-Substitution Devices

  • Other examples of sensory-substitution devices include:

    • The VOICE: Converts visual information to sound.

    • Other devices for the deaf and burn victims, translating stimuli into tactile sensations.

  • Understanding from these devices suggests that perceptual information can be processed similarly irrespective of sensory input.

Historical Context

  • Paul Bach-y-Rita: Pioneer of sensory-substitution technology; developed the first device in 1969.

  • Initial prototype: A bulky device that translated visual information into tactile sensations using a very different mechanism than current devices.

  • Neuroplasticity concept: The brain’s ability to adapt even after loss of sensory functions, indicated by Bach-y-Rita’s work.

Current Applications and Challenges

  • BrainPort has received FDA approval, but the high cost (about $10,000) and limited user base hinder widespread adoption.

  • Training and familiarization with the device is intensive—users need to practice for at least 20 minutes daily to gain proficiency.

  • User Experience: Many blind individuals may not find the device worthwhile compared to the effort and expense involved in learning to use it.

Sensory Knowledge and Brain Function

  • Recent studies show that congenitally blind people can learn to use devices like the vOICe to gain visual perception-like information.

  • The human brain exhibits remarkable plasticity, which challenges previous theories about critical periods in sensory development.

Theoretical Implications

  • Sensory-substitution devices prompt discussions among philosophers about the nature of perception and whether sensations derived from these devices can truly be considered vision.

  • Philosophical debate centers around:

    • Definition of vision based on the sensory organ or the type of stimulus processing involved.

    • Reports from users on perceived experiences varying widely, influencing the understanding of how sensory modalities are defined.

Conclusion

  • Technological strides in sensory substitution reframe our understanding of brain capabilities and sensory experiences, offering new possibilities for rehabilitation and enhancement of sensory perceptions.

  • Weihenmayer describes using the BrainPort as regaining a sense of normalcy, especially in social contexts with family.