The Comparison of K-Nearest Neighbors and Random Forest Algorithm to Recognize Indonesian Sign Language in a Real-Time
(1) Department of Computer Science, STMIK ESQ, Indonesia
(2) Department of Computer Science, STMIK ESQ, Indonesia
Abstract
Purpose: Comparing 2 models or prototype programs which can recognize Indonesian Sign Language System or Sistem Isyarat Bahasa Indonesia (SIBI) fonts from hand gesture and translate it’s into writing Messages in real-time.
Methods: After selecting datasets and reprocessed by the researcher into 1 dataset, which are a combination of several sign image datasets of the SIBI letters images available on the Kaggle website, the dataset is converted into landmarks. The landmarks are divided into 26 sign classes and preprocessed to a total of 19,826 rows of data, and then divided into 67% training data and 33% test data. Next, both K-NN and Random Forest algorithm are implemented into different program and get tested into 2 different tests, model evaluation and real-time. At the end, the result is compared to see the increase of accuracy level of both K-Nearest Neighbors (K-NN) and Random Forest algorithm.
Result: The constructed and trained model is then evaluated and the results of Precision, Recall, Accuracy, and F1-Score are 99.88% using the Random Forest algorithm. The results of real-time program testing with the K-Nearest Neighbors algorithm get higher results, where the average accuracy value reaches 99%.
Novelty: From the result shows that the model built with the Random Forest algorithm is superior, but the K-Nearest Neighbors algorithm is better in real-time testing. Therefore, image data and its diversity should be increased, in order to improve recognition accuracy. The program could be enhanced by adding a function where the program can recognize hand gesture, not only one or two hands but also can recognize a hand gesture with movements so the program can recognize static and dynamic letter (required hands movement).
Keywords
Full Text:
PDFReferences
S, A. Potluri, S. M. George, G. R and A. S, "Indian Sign Language Recognition Using Random Forest Classifier," 2021 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT), Bangalore, India, 2021, pp. 1-6, doi: 10.1109/CONECCT52877.2021.9622672.
Su, R.; Chen, X.; Cao, S.; Zhang, X. “Random Forest-Based Recognition of Isolated Sign Language Subwords Using Data from Accelerometers and Surface Electromyographic Sensors”. Sensors 2016, 16, 100. https://doi.org/10.3390/s16010100.
Kenshimov, Chingiz, et al. "Sign Language Dactyl Recognition Based on Machine Learning Algorithms." Eastern-European Journal of Enterprise Technologies, vol. 4, no. 2, 2021, pp. 58-72, doi:10.15587/1729-4061.2021.239253.
Yugam Bajaj and Puru Malhotra “American Sign Language Identification Using Hand Trackpoint Analysis” Springer, arxiv:2010.10590
Mahalakshmi V “Sign Language Training Tool Using Machine Learning Techniques” International Journal of Research Publication and Reviews, Vol 4, no 6, pp 3488-3494 June 2023
Sunanda Das, et al “A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier” Elsevier: Expert Systems with Applications Volume 213, Part B, 1 March 2023, 118914. https://doi.org/10.1016/j.eswa.2022.118914
Radha S. Shirbhate, et al “Sign language Recognition Using Machine Learning Algorithm” International Research Journal of Engineering and Technology (IRJET) Volume: 07 Issue: 03 Mar 2020
Rasha Amer Kadhim and Muntadher Khamees “A Real-Time American Sign Language Recognition System using Convolutional Neural Network for Real Datasets” TEM Journal. Volume 9, Issue 3, Pages 937-943, ISSN 2217-8309, DOI: 10.18421/TEM93-14, August 2020
Hisham, B., Hamouda, A. “Supervised learning classifiers for Arabic gestures recognition using Kinect V2”. SN Appl. Sci. 1, 768 (2019). https://doi.org/10.1007/s42452-019-0771-2
Jitendra Jaiswal “A Comparative Analysis On Sign Language Prediction Using Machine Learning Algorithms” 2020, https://api.semanticscholar.org/CorpusID:219616308
Fitri Utaminingrum, et al “Alphabet Sign Language Recognition Using K-Nearest Neighbor Optimization”. https://www.semanticscholar.org/ DOI:10.17706/jcp.14.1.63-70, Corpus ID: 59616174
Arsheldy Alvin, et al “Hand Gesture Detection for American Sign Language using K-Nearest Neighbor with Mediapipe”, 2021, https://api.semanticscholar.org/CorpusID:247358722
Neeraj Kumar Pandey, et al “An Improved Sign Language Translation approach using KNN in Deep Learning Environment” https://api.semanticscholar.org/DOI:10.1109/ICDT57929.2023.10150934, Corpus ID: 259217415
Madhuri Sharma, et al “Indian Sign Language Recognition Using Neural Networks and K-NN Classifiers” ARPN Journal of Engineering and Applied Sciences VOL. 9, NO. 8, AUGUST 2014 ISSN 1819-6608
Dewinta and Yaya Heriyadi, “American Sign Language-Based Finger-spelling Recognition using k-Nearest Neighbours Classifier” Conference: The 3rd International Conference on Information and Communication Technology, Bali, Indonesia, May 2015, DOI:10.1109/ICoICT.2015.7231481. https://www.researchgate.net/publication/279198249_American_Sign_Language-Based_Finger-spelling_Recognition_using_k-Nearest_Neighbours_Classifier
M. Rajanishree, N. Nadeem Ahmed, Y. Panchani, S. Aravindan and V. Jadhav, "Sign Language Conversion to Speech with the Application of KNN Algorithm," 2022 Sixth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Dharan, Nepal, 2022, pp. 886-890, doi: 10.1109/I-SMAC55078.2022.9987421.
N. B. Linsangan, J. V. G. Calites, J. T. L. Reyes, G. C. D. Sioson, R. V. Pellegrino and I. C. Juanatas, "Filipino Sign Language to Text Converter using K-Nearest Neighbor Algorithm," 2022 IEEE 14th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM), Boracay Island, Philippines, 2022, pp. 1-6, doi: 10.1109/HNICEM57413.2022.10109512.
S. Sharma, R. Sreemathy, M. Turuk, J. Jagdale and S. Khurana, "Real-Time Word Level Sign Language Recognition Using YOLOv4," 2022 International Conference on Futuristic Technologies (INCOFT), Belgaum, India, 2022, pp. 1-7, doi: 10.1109/INCOFT55651.2022.10094530.
Malek Zakarya Alksasbeh, et al “Smart hand gestures recognition using K-NN based algorithm for video annotation purposes” The Indonesian Journal of Electrical Engineering and Computer Science (IJEECS), p-ISSN: 2502-4752, e-ISSN: 2502-4760, DOI: http://doi.org/10.11591/ijeecs.v21.i1.pp242-252
Fifin Ayu Mufarroha and Fitri Utaminingrum “The Greatest Points in Hand Gesture Recognition for American Sign Language” International Journal of Intelligent Engineering and Systems, 2017
I.A. Adeyanju , et al “Machine learning methods for sign language recognition: A critical review and analysis” Intelligent Systems with Applications Volume 12, November 2021, 200056. https://doi.org/10.1016/j.iswa.2021.200056
Lester Wong Sze Ee, et al “Real-Time Sign Language Learning System” Journal of Physics: Conference Series 1712 (2020) 012011 IOP Publishing doi:10.1088/1742-6596/1712/1/012011
Ahmed Sultan, et al “Sign language identification and recognition: A comparative study” journal Open Computer Science. https://doi.org/10.1515/comp-2022-0240
Rupesh Kumar, et al “A Comparative Analysis of Techniques and Algorithms for Recognising Sign Language” Arxiv: 2305.13941
Hany A. AbdElghfar, Abdelmoty M. Ahmed, Ali A. Alani, Hammam M. AbdElaal, Belgacem Bouallegue, Mahmoud M. Khattab, Gamal Tharwat, Hassan A. Youness, "A Model for Qur’anic Sign Language Recognition Based on Deep Learning Algorithms", Journal of Sensors, vol. 2023, Article ID 9926245, 13 pages, 2023. https://doi.org/10.1155/2023/9926245
Zafar Ahmed Ansari and Gaurav Harit “Nearest neighbour classification of Indian sign language gestures using kinect camera” Sadhana Vol. 41, No. 2, February 201, pp. 161–182
Zahid H, Rashid M, Syed SA, Ullah R, Asif M, Khan M, Abdul Mujeeb A, Haider Khan A. 2022. A computer vision-based system for recognition and classification of Urdu sign language dataset. PeerJ Comput. Sci. 8:e1174 http://doi.org/10.7717/peerjcs.1174
Nurhadi, “Mengenal Bisindo dan Sibi, 2 Bahasa Isyarat yang Digunakan di Indonesia - Difabel Tempo.co,” Tempo.co. Accessed: Oct. 06, 2022. [Online]. Available: https://difabel.tempo.co/read/1624137/mengenal-bisindo-dan-sibi-2-bahasa-isyarat-yang-digunakan-di-indonesia.
N. Aziz and A. Kurniawardhani, “The Development of Hand Gestures Recognition Research: A Review,” International Journal of Artificial Intelligence Research, vol. 6, no. 1, Jun. 2021, doi: 10.29099/ijair.v6i1.236.
K. Taunk, S. De, S. Verma, and A. Swetapadma, A Brief Review of Nearest Neighbor Algorithm for Learning and Classification. IEEE, 2019.
Dertat, “Applied Deep Learning - Part 4: Convolutional Neural Networks,” Medium. Accessed: Jan. 13, 2023. [Online]. Available: https://towardsdatascience.com/applied-deep-learning-part-4-convolutional-neural-networks-584bc134c1e2.
F. Damatraseta, R. Novariany, and M. A. Ridhani, “Real-time BISINDO Hand Gesture Detection and Recognition with Deep Learning CNN,” Jurnal Informatika Kesatuan, vol. 1, no. 1, pp. 71–80, Jul. 2021, doi: 10.37641/jikes.v1i1.774.
M. Z. Alksasbeh et al., “Smart hand gestures recognition using K-NN based algorithm for video annotation purposes,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 21, no. 1, pp. 242–252, Jan. 2021, doi: 10.11591/ijeecs.v21.i1.pp242-252.
P. E. Wiraswendro and H. Soetanto, “Penerapan Algoritma Random Forest Classifier Pada Sistem Deteksi Simbol Sistem Isyarat Bahasa Indonesia (SIBI),” 2022. [Online]. Available: https://pmpk.kemdikbud.go.id/sibi/
“Kamus SIBI.” Accessed: Jun. 04, 2023. [Online]. Available: https://pmpk.kemdikbud.go.id/sibi/kosakata
L. Afifah, “Algoritma K-Nearest Neighbor (KNN) untuk Klasifikasi - IlmudataPy.” Accessed: Jan. 14, 2023. [Online]. Available: https://ilmudatapy.com/algoritma-k-nearest-neighbor-knn-untuk-klasifikasi/
L. Breiman, “Random Forests,” 2001. “What is Random Forest? | IBM,” IBM. Accessed: May 22, 2023. [Online]. Available: https://www.ibm.com/topics/random-forest.
Tharwat, “Classification assessment methods,” Applied Computing and Informatics, vol. 17, no. 1, pp. 168–192, 2018, doi: 10.1016/j.aci.2018.08.003.
Markoulidakis, G. Kopsiaftis, I. Rallis, and I. Georgoulas, “Multi-Class Confusion Matrix Reduction method and its application on Net Promoter Score classification problem,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Jun. 2021, pp. 412–419. doi: 10.1145/3453892.3461323.
Liao, Xianghua & Zheng, Jiaxuan & Huang, Chengli & Huang, Guoru. (2018). Approach for Evaluating LID Measure Layout Scenarios Based on Random Forest: Case of Guangzhou—China. Water (Switzerland). 10. 10.3390/w10070894.
Refbacks
- There are currently no refbacks.
Scientific Journal of Informatics (SJI)
p-ISSN 2407-7658 | e-ISSN 2460-0040
Published By Department of Computer Science Universitas Negeri Semarang
Website: https://journal.unnes.ac.id/nju/index.php/sji
Email: [email protected]
This work is licensed under a Creative Commons Attribution 4.0 International License.