NP Allies Return to NPA Main Site

Treatment overview

Sensory Substitution

Sensory-substitution approaches convert visual information into another sensory modality—usually sound or touch—so that people with vision loss can gain functional information about their environment. Training helps the brain learn to interpret the substituted signals for navigation, object recognition, or other daily tasks.

Overview

What It Is Sensory substitution is an approach that routes information from a missing or impaired sense through a different, intact sense. A camera, microphone, or other sensor captures the environment; software converts that information into patterns of touch, sound, or another modality that the brain can learn to interpret. With training, many users develop functional awareness—such as locating objects, reading text, or detecting sound sources—without restoring the original sense itself.

How It Works The brain can reorganize so that sensory cortex normally used for one modality begins to process information arriving through another channel (cross-modal plasticity). For example, visual information delivered as patterns on the tongue or skin, or as structured sound, can be interpreted as spatial and object information after practice. Devices do not recreate natural vision or hearing; they provide a usable substitute channel that the nervous system learns to use.

How It Is Used Today (leading applications) - Vision substitution for blindness or severe low vision — camera-to-tongue, camera-to-skin, or camera-to-audio systems that support orientation, mobility, and object recognition - Hearing / sound awareness substitution — wrist or torso wearables that convert sound into vibration patterns for people with hearing loss - Balance and spatial orientation support — tongue or skin stimulation that provides supplemental postural information - Reading and face/text recognition — wearable cameras that speak text, identify faces, or describe scenes aloud

Example technologies and interventions - BrainPort Vision Pro — camera-to-tongue electrotactile display for functional environmental awareness in blindness - Tongue display units (TDUs) — research and clinical systems (including BrainPort-class devices) that map images to patterns on the tongue - OrCam MyEye — wearable camera that provides spoken feedback for text, faces, products, and scene information - The vOICe — visual-to-auditory sensory substitution (image sonification) used in research and by some independent users - EyeMusic and related sonification systems — alternative visual-to-audio mappings for spatial and object information - Other emerging haptic vests, belts, and AI-enhanced description tools that expand substitution and sensory-augmentation options

On-label / regulatory status - These are primarily assistive devices, not disease-modifying therapies. Regulatory status varies by product (some have FDA clearance as assistive technology; many consumer systems are not prescription medical devices). - They are used to support function in vision or hearing loss rather than as approved cures for eye or ear disease. - Training-dependent; not a substitute for ophthalmology or audiology care when those are indicated.

Conditions - Primary uses: vision loss (functional awareness and mobility support) and hearing loss (sound awareness) - Also applied for: balance deficits and selected sensory-processing support needs - Note: These tools are generally assistive and training-dependent, not cures that restore natural sight or hearing.

How it is administered

Depends on the specific system: wearable cameras with tongue, skin, or audio output; wrist or torso vibration devices; or software-based sonification. Most require an initial training period (hours to weeks) so the brain can learn the new sensory mapping, then daily use in real environments.

The NPA Treatment Database lists this treatment in connection with 3 condition records. A database relationship does not establish effectiveness, safety, or suitability for an individual.

These controlled question pages reference this treatment through stored condition relationships.

Symptoms connected in the NPA database

The current record associates this treatment with the following symptom information. This is not a claim that the treatment will improve these symptoms.

Cognition & Memory

  • Delayed cognitive skill development
  • difficulty concentrating
  • mental fog
  • Short-term memory issues
  • Short-term memory problems
  • slowed processing
  • slowed thinking
  • spatial neglect

Motor Control

  • Balance and coordination problems
  • Delayed gross or fine motor milestones
  • difficulty walking
  • muscle stiffness/spasticity
  • One-sided weakness or paralysis
  • poor balance and coordination
  • reduced fine motor skills
  • slowed movements
  • weakness

Sensory & Pain

  • central post-stroke pain (burning, shooting pain, and sensitivity to touch, temperature)
  • headaches
  • Numbness or altered sensation
  • numbness or tingling
  • Persistent headaches
  • sensitivity to light/noise
  • tinnitus
  • Variable sensory processing differences

Visual & Auditory Processing

  • Blurred or double vision
  • difficulty processing sounds
  • difficulty processing visual information
  • double vision
  • light sensitivity
  • Possible delays in visual-motor or auditory processing skills
  • Visual field loss

Practical information

Insurance coverage

Varies widely. Some low-vision and hearing assistive devices may qualify for coverage or vocational/rehab funding; many consumer sensory-substitution products are out-of-pocket.

Cost information

Wide range: from lower-cost software/apps to several thousand dollars for clinical-grade devices (e.g., BrainPort, OrCam) plus training.

Results and outcomes described in the source record

Basic awareness can emerge within hours to days of training; more fluent functional use typically develops over weeks of consistent practice.

Side effects and risks

Generally low risk. Possible local irritation (tongue or skin), sensory fatigue, or cognitive load during early learning. Not a replacement for medical eye or ear care when those are indicated.

Research context in the NPA database

This section reproduces the current database's research summary and detail. Structured citation metadata has not been supplied, so it should not be read as an independently verified literature review.

Research summary

Evidence Level: Emerging to Moderate (varies by device) What the research shows: Feasibility and clinical studies support that trained users can gain usable environmental information through tongue, skin, or auditory channels. Effect sizes and real-world benefit depend on the device, training quality, and user goals. Key takeaway: A family of assistive approaches that leverage cross-modal brain plasticity; best viewed as tools for functional awareness rather than restoration of natural vision or hearing.

Research detail

Overview Sensory substitution delivers information from an impaired sense through another intact sense (touch, hearing, etc.), relying on cross-modal neuroplasticity and user training.

Evidence Base and Published Work • Foundational work by Bach-y-Rita and colleagues established that the brain can interpret visual information delivered via tactile channels. • Clinical and feasibility studies of BrainPort-class tongue displays show improved object recognition and orientation-and-mobility task performance after training. • Auditory and vibrotactile substitution (e.g., Neosensory-style devices, The vOICe, EyeMusic) have supporting user and research literature for sound awareness and visual-to-audio mapping. • Wearable camera systems such as OrCam provide practical text, face, and scene information via speech output. • Larger standardized trials and insurance pathways remain limited for many devices; outcomes depend heavily on training and consistent use.

Outcomes Outcomes most frequently described include improved environmental awareness, object or text access, and mobility confidence after training. These systems complement rather than replace residual vision, hearing aids, or other clinical care.

Questions to ask a provider

1. Is a sensory-substitution approach appropriate for my type of vision or hearing loss, and what are realistic functional goals? 2. Which technology fits me best (tongue display, wearable camera with speech, sound-to-vibration, sonification app)? 3. How much training is required, and who provides it? 4. What is the cost, and is any funding available through vocational rehab, insurance, or assistive-technology programs? 5. How does this compare or combine with other low-vision or hearing tools I already use?

Website and contact information

Explore the interactive Treatment Database

Continue to compare educational treatment information in the interactive database. The standard disclaimer acknowledgement is required before the database opens.

Explore treatment information in the database

Educational information disclaimer

The information provided in this app is for educational and informational purposes only. It is not intended as medical advice, diagnosis, or treatment, and should not be relied upon as a substitute for professional medical care.

Treatments, therapies, and interventions described here may not be appropriate for every person or condition. Evidence levels, availability, costs, insurance coverage, and risks can vary. Nothing in this app constitutes a recommendation that you start, stop, or change any treatment.

Always consult your physician, neurologist, or other qualified healthcare provider before beginning any new therapy, device, program, or intervention, and before making decisions about your care. Seek prompt medical attention for any urgent or emergency symptoms. By using this app, you acknowledge that NP Allies and its contributors are not responsible for any decisions you make based on the information provided, or for any outcomes that result from those decisions.