Rivaldo Nugraha, Rofik Adam Nugraha, Muhammad Abdul Mujib, Okyza Maherdy Prabowo, Anggi Cipta Lestari, Shafira Febriani

DETEKSI ABJAD BAHASA ISYARAT BISINDO REAL-TIME PADA SMARTPHONE MENGGUNAKAN YOLOV11

Introduction

Deteksi abjad bahasa isyarat bisindo real-time pada smartphone menggunakan yolov11. Deteksi abjad Bahasa Isyarat BISINDO A-Z real-time di smartphone pakai YOLOv11. Solusi komunikasi inklusif bagi tunarungu & gangguan pendengaran dengan AI & ML.

Abstract

Bahasa isyarat merupakan sarana komunikasi utama bagi komunitas tunarungu dan penyandang gangguan pendengaran, sekaligus berperan penting dalam meningkatkan inklusivitas sosial dan menjembatani kesenjangan komunikasi dengan masyarakat umum. Seiring meningkatnya jumlah penyandang gangguan pendengaran, diperlukan solusi komunikasi yang mudah diakses dan bersifat adaptif. Perkembangan kecerdasan buatan dan pembelajaran mesin memberikan peluang untuk mengotomatisasi deteksi bahasa isyarat secara real-time. Penelitian ini mengembangkan sistem deteksi abjad BISINDO A–Z menggunakan dua pendekatan, yaitu YOLOv11 dan RF-DETR. YOLOv11 dipilih karena arsitekturnya ringan dan efisien untuk perangkat mobile, sedangkan RF-DETR digunakan sebagai pembanding berbasis Transformer dengan tingkat akurasi tinggi. Hasil eksperimen menunjukkan bahwa RF-DETR mencapai akurasi lebih tinggi dengan mAP@50 sebesar 99,8%, namun terbatas pada implementasi berbasis web. YOLOv11 memperoleh mAP@50 sebesar 99,4% dan berhasil diimplementasikan pada smartphone Android dengan kinerja real-time yang responsif. Temuan ini menunjukkan bahwa YOLOv11 lebih layak untuk aplikasi deteksi abjad BISINDO berbasis perangkat mobile.

Review

This paper presents a highly relevant and impactful contribution to the field of assistive technology, focusing on bridging communication gaps for the deaf and hard-of-hearing community. By addressing the critical need for accessible and adaptive communication solutions, the authors leverage recent advancements in artificial intelligence and machine learning to develop a real-time system for detecting the A-Z alphabet of BISINDO (Bahasa Isyarat Indonesia) sign language. The initiative is particularly commendable for its focus on mobile implementation, aiming to provide a practical tool for improving social inclusivity and everyday communication. The research employs a comparative approach, evaluating two prominent deep learning architectures: YOLOv11 and RF-DETR. YOLOv11 was specifically chosen for its efficient and lightweight design, making it ideal for smartphone deployment. RF-DETR, a Transformer-based model recognized for its high accuracy, served as a high-performance benchmark. Experimental results demonstrate that while RF-DETR achieved a superior mAP@50 of 99.8%, its implementation was limited to web-based platforms. Crucially, YOLOv11, despite a marginally lower mAP@50 of 99.4%, successfully demonstrated real-time and responsive performance when deployed on Android smartphones, thereby fulfilling the core objective of a mobile-centric solution. The primary strength of this work lies in its successful real-time implementation of a high-accuracy sign language detection system on a smartphone, directly addressing a significant practical need. The achieved accuracy of YOLOv11 (99.4% mAP@50) on a mobile device is highly promising for real-world application. While the abstract clearly outlines the trade-off between accuracy and mobile deployability, the authors effectively justify YOLOv11 as the more viable solution for a smartphone-based system. This research represents a valuable advancement in creating practical tools for the deaf community and holds significant potential for enhancing social inclusivity. Future directions could explore expanding detection capabilities beyond individual alphabets to full words and phrases, as well as optimizing other high-accuracy models for mobile environments.

Full Text

Comments

You need to be logged in to post a comment.