Open Access Open Access  Restricted Access Subscription Access

Real-Time Vision-Based Sign Language Recognition Using Mediapipe and CNN

Sri Ragavendhira S, Vishnu Ram M, Yogesh P, Sanjaykumar S, Ms. Sini Prabhakar

Abstract


Communication barriers between hearing and speech-impaired individuals and the general population often hinder effective interaction and social inclusion. Traditional approaches to sign language interpretation rely heavily on human translators, which can be costly, inconsistent, and unavailable in real-time scenarios. With advancements in computer vision and deep learning, automated sign language recognition offers a promising solution to bridge this communication gap.

This project presents a vision-based Sign Language Recognition (SLR) system that interprets hand gestures captured through a standard webcam into corresponding text or speech output. The system utilizes OpenCV for image preprocessing, Mediapipe for real-time hand landmark detection, and a Convolutional Neural Network (CNN) model for gesture classification. The framework is designed to recognize alphabets and commonly used signs from Indian Sign Language (ISL) with high accuracy and responsiveness.

Experimental results show that the proposed system achieves reliable gesture recognition under varying lighting and background conditions. It provides an efficient, low-cost, and accessible solution that promotes inclusive communication, with potential applications in assistive technology, education, and human–computer interaction.


Full Text:

PDF

References


• Zhang, Z. (2012). Microsoft Kinect sensor and its effect. IEEE Multimedia, 19(2), 4–10.

• Lugaresi, C., et al. (2019). Mediapipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172.

• Supriya, M., & Shreya, B. (2021). Hand gesture recognition using computer vision. International Journal of Computer Applications, 174(30), 12–17.

• Patel, H., & Sharma, V. (2020). Sign language recognition using OpenCV and CNN. Journal of Emerging Technologies, 8(2), 45–52.

• PyAutoGUI Documentation (2024). Retrieved from: https://pyautogui.readthedocs.io/

• Mittal, A., & Aggarwal, A. (2018). Real-time hand gesture recognition using OpenCV. International Journal of Computer Science and Information Technologies, 9(2), 1–5.

• Ghotkar, A. S., & Kharate, G. K. (2013). Hand gesture recognition for Indian Sign Language. International Conference on Computer Communication and Informatics (ICCCI), IEEE.

• Molchanov, P., Gupta, S., Kim, K., & Kautz, J. (2015). Hand gesture recognition with 3D convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

• OpenCV Documentation (2024). Open Source Computer Vision Library. Retrieved from: https://opencv.org/

• Rautaray, S. S., & Agrawal, A. (2015). Vision-based hand gesture recognition for human–computer interaction: A survey. Artificial Intelligence Review, 43(1), 1–54.


Refbacks

  • There are currently no refbacks.