Sign language is a language that is completely visual and uses gestures, as well as hand shapes, to represent concepts or ideas. Sign language is actually a broad term that describes many visual languages that may have different grammar and syntax rules, but uses the same basic signs for common words. At one end of the sign language spectrum is American Sign Language. ASL is a real language (like English) with it’s own grammar and syntax.
The National Center for Health Statistics estimates that 28 million Americans (about 10% of the population) have some degree of hearing loss. About 2 million of these 28 million people are classified as deaf (they are unable to hear everyday sounds or speech even with a hearing aid), according to StartASL.
September 23rd Was the First International Day of Sign Languages
On September 23rd, 2018, the United Nations marked the first International Day of Sign Languages, as part of the International Week of the Deaf. First celebrated in September 1951, the International Week of the Deaf has since evolved into a global movement of deaf unity and concerted advocacy to raise awareness of the issues deaf people face in their everyday lives.
The World Federation of the Deaf strongly believes that the present United Nations resolution is a valuable extension of this tradition and has the potential to increase the understanding of United Nations member states, the private sector and the United Nations’ system in closing existing gaps in the achievement of human rights for deaf people.
Bringing Sign Language to Virtual and Augmented Reality
With technology like Google Translate, we can communicate in almost any language in the world, even if we do not know that language. However, there’s one group of people who are left out, and that is deaf and hard of hearing. Many people are trying to address this gap by leveraging virtual and augmented reality technology.
Learning and Promoting Sign Language
Three NYU Tandon School of Engineering graduate students recently developed an application that translated sign language into readable text. This mobile app is called “ASRL – Augmented Reality Sign Language”, and is able to use computer vision and augmented reality to translate between different versions of sign language, as well as between spoken language and sign.
The app allows a user to sign, turning this into text and speech for the non-signer to understand. The non-signer can then speak into the app which will translate the spoken speech into sign language for the hearing impaired user using augmented reality.
In 2016, the HoloHear team also created a prototype augmented reality app at a Microsoft HoloLens Hackathon in San Francisco. It also translated spoken word into sign language, as well as taught sign language, and was created using specifically the HoloLens. When a user would speak aloud, a 3D holographic model appeared to translate the sentence in American Sign Language in real time.
Limited in its usefulness for mass deployment, it was more designed to demonstrate how augmented reality can help us communicate better with one another.
KinderSign, is an interactive mobile app created by AmphibiStudio, that uses augmented reality to teach Singapore Sign Language (SGSL) to children. The platform teaches children how to fingerspell and sign a particular word or phrase using digital flashcards.
The virtual reality game, Moss, was first announced by Playstation VR at Sony’s E3 2017. The protagonist of the game, Quill, is a mouse character that not only conveys personality through movement and body language, but also signs. This was a huge step for the deaf and hearing impaired, both adults and children, by seeing positive awareness being included within a video game.
The Future of Language and Technology Unification
On Sunday, September 30th, we also celebrated the first International Translation Day. The United Nations declares this day as an opportunity to pay tribute to the work of language professionals, which plays an important role in bringing nations together, facilitating dialogue, understanding and cooperation, contributing to development and strengthening world peace and security.
Transposition of a literary or scientific work, including technical work, from one language into another language, professional translation, including translation proper, interpretation and terminology, is indispensable to preserving clarity, a positive climate and productiveness in international public discourse and interpersonal communication. This can only be amplified by the ability for these works to transcend barriers, and this can seemingly be done with virtual and augmented reality technology.
Together, through the continued evolution of virtual and augmented reality, we can continue to embrace and support many people trying to bring the world together through the power of language.
Technology can sometimes seem like a tool of isolation, just how similarly the deaf and hearing impaired may feel in their ability to communicate with the world. However, emerging virtual and augmented reality can offer ways to aide in human interaction, proving its continued importance in helping us to communicate better with one another.
