Friday, October 19, 2018
ApplicationAugmented RealityInclusivitySocial issuesVirtual Reality

Inclusivity of VR and AR Accessibility for the Visually and Hearing Impaired

While it seems VR and AR technology could make global accessibility difficult for those with hearing and visual impairments, companies are exploring the potential solutions for a more inclusive experience.

VR and AR provide the amazing ability to explore new worlds, merge the digital world with the real world, and go on all sorts of adventures without even having to leave home. They also offer amazing opportunities for a multitude of different people with disabilities, and even quality of life benefits. People love the sense of exploration, entertainment, aid, and freedom that VR and AR offer, but it’s also a type of technology that relies on visual and audio cues. This brings new barriers to those who may have vision or hearing impairments, but also opens up innovative ways to leverage VR and AR to not only explore solutions in entertainment for all, but creative solutions to assist the visual or hearing impaired in new and improved ways.

See Also:  How Augmented Reality Has Changed Life For the Disabled

Knowing these challenges, some companies have set out to explore alternative ways to offer VR and AR experiences in a way that’s accessible to those who are vision and hearing impaired. However, there have already been some extraordinary breakthroughs in an exciting new world that is still in its infancy.

 

VR and AR Accessibility for the Visually Impaired

Google Daydream is one major company testing the use of spatial audio cues for those with visual impairments. Using a prototype of a virtual room, Daydream Labs recorded and linked the name of every object in the room, and to enable navigation in a pitch-black room, created a 3D audio laser pointing system from an HTC Vive controller. This would then select and play the audio labels while leveraging audio location control. This allows the system to play a short impulse response tone at the location of the controller that quickly progresses to the actual distance to the virtual object.

VR/AR Accessibility for the Visually Impaired Google Daydream
Source: Google

Similarly, Microsoft created the Canetroller, which essentially takes the concept of a blind  or visually impaired individual’s white cane, which is a tool used to detect surrounding obstacles in their environment, and applies it to the world of virtual reality.

The Canetroller relies on haptic and audio feedback to simulate the world around the user in a virtual world. This is achieved by the five essential parts of the Canetroller itself. This is a braking mechanism, a controller, a slider that connects the braking mechanism to the cane, and a voice coil that’s mounted on the cane to provide the sensation of vibration to the user. The HTC Vive is actually mounted to the controller itself.

canetroller
Source: Microsoft

There are also even applications being developed to bring inclusivity to VR and AR technology. Near Sighted VR Augmented Aid is an Android mobile application that uses a user’s mobile device camera and displays the image to the phone screen in stereoscopic view. It is most beneficial for people who have visual impairments such as Stargardt’s disease, macular degeneration, optic atrophy, and more.

Additionally, companies like Aira, leverage augmented reality through a subscription based service that provides the ability to live stream video from smart glasses, such as Google Glass. Their focus is to assist visually-impaired people navigate their environments through a trained professional agent, at the touch of a button. Erich Manser, who is legally blind, was helped through the 26.2 mile Boston Marathon course using Aira’s assistive technology.

Aira
Source: Aira
Touching Masterpieces VR and AR
Source: Touching Masterpieces

Experiencing the beauty of art has always relied on the ability to actually visually see these masterpieces of the world. Not anymore. From the creative collaboration team of Geometry Prague, NeuroDigital Technologies, and the Leontinka Foundation for the blind and visually impaired, art is now accessible to the blind through virtual reality technology. This real-life exhibit has been called Touching Masterpieces.

Users just put on a pair of NeuroDigital’s haptic gloves and an HMD, and they are free to explore recreated avatars of the most famous sculptures in the world, such as Michelangelo’s David, or the bust of Nefertiti in complete virtual reality. The original artworks were first mapped with laser scans, and added multiple layers of details that encompass the real-life statues. This results in a realistic tactile experience that allows users to feel these statues.

 

Augmented Reality for the Hearing Impaired

Augmented reality already bridges the gap between the digital and the physical, so it seems only logical that it should bridge the gap between the digital and accessibility for the hearing impaired.

One design company from Germany, Morphoria, created Deaf Magazine, which was in publication for a limited time, and focused on the culture of German Sign Language, that combined with their own augmented reality application. This let readers interact with the magazine’s pages by learning new words in sign, or even teach those who do not understand sign.

Also in Germany, Greta & Starks are iOS and Android applications that allow those with vision or hearing impairments to still enjoy the movie experience via smart glasses. Greta speaks descriptions and actions in time with the movie being played, and Starks displays captions in a variety of different languages.

Greta & Starks VR and AR
Source: Greta & Starks

London is following a similar feat with London’s National Theatre using Epson’s latest smart glasses which display subtitles in user’s field of vision. The National Theatre already had set up caption screens, and it reserves a set of seats specifically with a view of them. However, this requires theater goers to look away from the stage in order to read the text. With Epson’s smart glasses, the subtitles are always within the wearer’s field of vision as they continue to watch performances with sacrificing missing any of the action.

 

A Future of Innovation to Make VR and AR Experiences More Accessible

There are plenty of hardware and software solutions to get excited about in which companies are ensuring they are creating VR and AR experiences that are accessible, and also have a positive effect.

It’s very easy to design these experiences for a world we are accustomed to living in, in the way we are used to experiencing it. It takes great effort to go beyond that and move VR and AR technology towards mainstream adoption that’s inclusive of everyone. Luckily, we have seen some great steps forward in these innovations to embrace not only VR and AR experiences, but also the human experience.

See Also:  World’s First Virtual Reality Zip Wire Opens In London
Patricia Chang
the authorPatricia Chang
Patricia Chang is a South Florida-based freelance Digital Project Manager and XR Strategist. She is also a U.S. Navy veteran born and raised in Las Vegas, Nevada, and has also resided in the states of California, Hawaii, and New York. With her B.S. in Computer Information Systems and Master’s in Project Management, Patricia has a decade of experience working with businesses at strategic and operational levels from technology start-ups to major corporations. When not doing project-based initiatives, you can find Patricia obsessing over anything VR/AR related, including attending a VR development academy, in hopes to fine tune her future digital consultancy business.