Open Access Open Access  Restricted Access Subscription Access

Interactive Learning of Malayalam Sign Language with Live Recognition in a Flutter Application

Sandeep Chandran, Anna Rose Bijoy, Anulekshmi A I, Anusha G P, Athira M S

Abstract


This paper presents a Flutter application designed to enhance accessibility and inclusivity for differently-abled individuals by facilitating the learning of Malayalam sign language. The app provides options for learning letters and words through videos sourced from the National Institute of Speech and Hearing (NISH). It also incorporates a live recognition feature using a Convolutional Neural Network (CNN) model for real-time recognition of sign language gestures. The backend of the app is implemented in Node.js, enabling seamless integration and efficient data processing. Additionally, the app includes practice tests to help users assess their learning progress. Overall, this app aims to provide an interactive and accessible platform for individuals to learn and communicate using sign language.


Full Text:

PDF

References


Indian Sign Language Alphabet in Malayalam - Released https://nish.ac. in/index.php/others/news/882-indian-sign-language accessed on march 2022.

Malayalam Sign Language Character Recognition

System https://ieeexplore.ieee.org/document/10245983

Sign language recognition of Malayalam alphabets using transfer learning https://ieeexplore.ieee.org/document/10142583

Malayalam sign language alphabets released by the Minister of Higher Education Dr. R Bindu https://www.reporterlive.com/newsroom/ malayalam-sign-language-alphabets-released-by-the-minister-of-higher -education-dr-r-bindu-60289

Pfister, T., Charles, J., & Zisserman, A. (2014). "Flowing ConvNets for Human Pose Estimation in Videos." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

Starner, T., & Pentland, A. (1997). "Real-time American Sign Language recognition from video using hidden Markov models." In Proceedings of the 3rd IEEE Workshop on Applications of Computer Vision

Li, Y., & Zhang, J. (2011). "Sign Language Recognition Using 3D Facial Motion Features." In Proceedings of the IEEE International Conference on Image Processing (ICIP).

Koller, O., & Ney, H. (2015). "Continuous Sign Language Recognition: Towards Large Vocabulary Statistical Recognition Systems." Machine Learning for Multimodal Interaction, Springer.

Kumud Tripathi, Neha Baranwal and G. C. Nandi, Continuous Indian Sign Language Gesture Recognition and Sentence Formation, Procedia Computer Science,vol.54,pp.523-531,December(2019)

Brandon Garcia and Sigberto Alarcon Viesca, Real-time American Sign Language Recognition with Convolutional Neural Networks, 1st International Cognitive Cities Conference (IC3). IEEE,pp. 202–203,September(2018).


Refbacks

  • There are currently no refbacks.