Arabic Sign Language Recognition: A Deep Learning Approach

Abstract

With more than 300 sign languages across the world, sign interprets are not always available to translate spoken words into sign language and vice versa. As people with hearing and speech impairments rely on Sign Language for communication, this would limit their communication with others. A solution for this would be utilizing Sign Language Recognition systems, which allow for communication between users of the sign language and those who do not without the need for interpreters. As we consider the success of Deep Learning for Computer Vision tasks, we observe the advantage it can provide for Arabic Sign Language Recognition. For this research, we have two aims. First, we would like to review the current status of research in Arabic Sign Language Recognition using Deep Learning and find research gaps. Second, we aim to build a Sign Language Recognition system that bridges the gap. We achieve this through a systematic review that identifies primary studies using deep learning models for Arabic Sign Language Recognition. Out of 414 identified studies, 67 were deemed of relevance to our topic. Out of those, 32 studies passed our full selection procedure. We were able to discover patterns in research and find that the biggest issue is data collection as current datasets don’t offer enough variety and are not representative of real-life scenarios. Current methods are either too expensive, or easily affected by the surrounding environment. Thus, for the second part, we offer a solution for data collection using MediaPipe, which allow us to collect data directly through the webcam. We are able to leverage this framework to build a recognition system for Emirati Sign Language that recognizes the signs for the seven Emirates. We used an LSTM model and achieve an accuracy of 100% in the testing dataset.