Sign Language Detection System Using YOLOv5 Algorithm to Promote Communication Equality People with Disabilities

Abstract

Purpose: Communication is an important asset in human interaction, but not everyone has equal access to this key asset. Some of us have limitations such as hearing or speech impairments, which require a different communicative approach, namely sign language. These limitations often present accessibility gaps in various sectors, including education and employment, in line with Sustainable Development Goals (SDGs) numbers 4, 8, and 10. This research responds to these challenges by proposing a BISINDO sign language detection system using YOLOv5-NAS-S. The research aims to develop a sign language detection model that is accurate and fast, meets the communicative needs of people with disabilities, and supports the SDGs in reducing the accessibility gap.

Methods: The research adopted a transfer learning approach with YOLOv5-NAS-S using BISINDO sign language data against a background of data diversity. Data pre-processing involved Super-Gradients and Roboflow augmentation, while model training was conducted with the Trainer of SuperGradients.

Result: The results show that the model achieves a mAP of 97,2% and Recall of 99.6% which indicates a solid ability in separating sign language image classes. This model not only identifies sign language classes but can also predict complex conditions consistently.

Novelty: The YOLOv5-NAS-S algorithm shows significant advantages compared to previous studies. The success of this performance is expected to make a positive contribution to efforts to create a more inclusive society, in accordance with the Sustainable Development Goals (SDGs). Further development related to predictive and real-time integration, as well as investigation of possible practical applications in various industries, are some suggestions for further research.