The Canadian Association of the Deaf reports that there are 357,000 culturally Deaf Canadians and 3.21 million hard of hearing Canadians, many of whom rely on American Sign Language (ASL) for communication (Canadian Association of the Deaf 2015). Culturally Deaf people are born into the deaf community and use sign language, regardless of their hearing ability, while individuals who are hard of hearing have a physical condition that limits their hearing. Methods of communication vary among members of the deaf community however, American Sign Language (ASL) is the primary language of many North Americans who are deaf and hard of hearing (National Institute on Deafness and Other Communication Disorders 2021). Advancements within the field of Artificial Intelligence (AI) show promise in facilitating sign language translation and education. Translation tools can be developed using large-scale databases of sensor recordings of sign language, which inform the creation and use of digital avatars capable of communicating in sign language.
Sign language capturing involves recording signs using sensors to create 3-dimensional digital representations of signs (Papastratis et al. 2021). Capturing sign language can occur in numerous ways. For example, an optical system with markers on the face can be used for tracking facial expressions, the posture of the signer can be tracked using a magnetic sensing body-suit, and gloves with built-in sensors can be used to determine hand and finger shape (Figure 1) (Havasi and Szabo 2005). Alternatively, optical systems where multiple high frame-rate cameras can be used to track the entire body (Havasi and Szabo 2005). The capturing of sign language data facilitates the creation of large datasets that can then be used to train and evaluate machine learning algorithms designed to recognize and translate sign language (Papastratis et al. 2021).
The final stage of integrating AI into sign language pertains to the rendering of sign language into 3D animations which can then be presented by digital avatars (Parton 2005). AI’s ability to automatically convert text into sign language would greatly increase the inclusion of the deaf community within predominantly hearing spaces. Additionally, it could help to facilitate the learning of deaf and hard of hearing students. A study conducted at the Florida School for the Deaf and Blind reported that story comprehension increased from 17% to 67% after seeing the story signed as opposed to being read (Parton 2005). When paired with current transcribing technologies, AI facilitated sign language representation shows promise in producing almost immediate conversion of English speech to sign language.
AI technology has many practical applications to promote the inclusion of deaf and hard of hearing individuals within environments of predominantly hearing individuals by enabling rapid translation and transliteration between both spoken, written and signed languages. As databases holding signed data continue to grow, these processes will become increasingly efficient and widely accessible. Ultimately, AI-powered tools have the potential to bridge communication gaps increasing accessibility for deaf and hard-of-hearing individuals in diverse settings.
References
Canadian Association of the Deaf. 2015. “Statistics on Deaf Canadians.” Canadian Association of the Deaf – Association Des Sourds Du Canada. July 3, 2015. https://cad-asc.ca/issues-positions/statistics-on-deaf-canadians/.
Havasi, L., and H.M. Szabo. 2005. “A Motion Capture System for Sign Language Synthesis: Overview and Related Issues.” In EUROCON 2005 – The International Conference on “Computer as a Tool,” 445–48. Belgrade, Serbia and Montenegro: IEEE. https://doi.org/10.1109/EURCON.2005.1629959.
Lu, Pengfei, and Matt Huenerfauth. 2014. “Collecting and Evaluating the CUNY ASL Corpus for Research on American Sign Language Animation.” Computer Speech & Language 28 (3): 812–31. https://doi.org/10.1016/j.csl.2013.10.004.
National Institute on Deafness and Other Communication Disorders. 2021. “What Is American Sign Language (ASL)? | NIDCD.” National Institute on Deafness and Other Communication Disorders. October 29, 2021. https://www.nidcd.nih.gov/health/american-sign-language.
Papastratis, Ilias, Christos Chatzikonstantinou, Dimitrios Konstantinidis, Kosmas Dimitropoulos, and Petros Daras. 2021. “Artificial Intelligence Technologies for Sign Language.” Sensors 21 (17): 5843. https://doi.org/10.3390/s21175843.
Parton, B. S. 2005. “Sign Language Recognition and Translation: A Multidisciplined Approach From the Field of Artificial Intelligence.” Journal of Deaf Studies and Deaf Education 11 (1): 94–101. https://doi.org/10.1093/deafed/enj003.