Open Access Open Access  Restricted Access Subscription Access

Autism Detection Using Self-Stimulatory Behaviors

Adithya Suresh, Faras A, Ummu Habeeba K M, Anu Eldho, Asha J George, Rotney Roy Meckamalil

Abstract


This paper introduces a novel video-audio-based model for the early detection of Autism Spectrum Disorder (ASD), focusing on analyzing self-stimulatory behaviors (stim- ming) such as arm flapping, head banging, and spinning, which are critical diagnostic markers. Traditional diagnostic approaches often depend on subjective clinical observations, leading to in- consistencies, delays, and limited accessibility in diverse settings. The proposed model combines video analysis with audio detection to address these shortcomings, supported by a generative AI- based method to create an audio dataset. This dataset enhances the model’s robustness by incorporating diverse audio features merged with video data to provide a comprehensive analysis. Video analysis employs YOLO for face detection, MediaPipe for facial landmarks, pose tracking, and gaze estimation, while CNN- LSTM models identify repetitive behaviors. Audio processing extracts features via Librosa, NoiseReduce, and PyDub, with Gradient Boosting models analyzing speech anomalies. Clas- sification integrates CNN-LSTM for video, Gradient Boosting for audio, and a Stacking Classifier with Logistic Regression for final predictions. By merging video and audio cues, this model enhances the objectivity, accuracy, and scalability of ASD detection, providing a reliable and efficient framework for early diagnosis and intervention across varied real-world environments

Full Text:

PDF

References


Md. Zasim Uddin, Md. Arif Shahriar, Md. Nadim Mahamood, Fady Alnajjar, Md. Ileas Pramanik, Md Atiqur Rahman Ahad, ”Deep learning with image-based autism spectrum disorder analysis: A systematic review”, Engineering Applications of Artificial Intelligence, Volume 127.

Rajagopalan, Shyam Sundar, Dhall, Abhinav, Goecke, Roland. (2013), ”Self-Stimulatory Behaviours in the Wild for Autism Diagnosis”, IC- CVW.

V. G. Prakash et al., ”Computer Vision-Based Assessment of Autistic Children: Analyzing Interactions, Emotions, Human Pose, and Life Skills,” in IEEE Access, vol. 11.

Rashid, Ali Hameed, Shaimaa. (2022). ”Review of Autistic Detection Using Eye Tracking and Vocalization Based on Deep Learning”. Journal of Algebraic Statistics. 13. 286-297.

Deng, Shijian, Erin E. Kosloski, Siddhi Patel, Zeke A. Barnett, Yiyang Nan, Alexander Kaplan, Sisira Aarukapalli, Matthew Wang, Harsh Singh, Pamela R. Rollins and Yapeng Tian. “Audio-Visual Autism Behavior Recognition With Multimodal Large Language Models.” Se- mantic Scholar.

A. Vyas, Bowen Shi, Matthew Le, Andros Tjandra and Wei-Ning Hsu (2023), ”Unified Audio Generation with Natural Language Prompts”arXiv

Han, Junxia et al. “A Multimodal Approach for Identifying Autism Spectrum Disorders in Children.” IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society vol. 30 (2022): 2003-2011.

S. Liang, A. Q. M. Sabri, F. Alnajjar and C. K. Loo, ”Autism Spectrum Self-Stimulatory Behaviors Classification Using Explainable Temporal Coherency Deep Features and SVM Classifier,” in IEEE Access, vol. 9

Ahmed, Z.A.T.; Albalawi, E.; Aldhyani, T.H.H.; Jadhav, M.E.; Janrao, P.; Obeidat, M.R.M. ”Applying Eye Tracking with Deep Learning Techniques for Early-Stage Detection of Autism Spectrum Disorders”. Data 2023.

W. Zhou, M. Yang, J. Tang, J. Wang, and B. Hu, ”Gaze Patterns in Chil- dren With Autism Spectrum Disorder to Emotional Faces: Scanpath and Similarity” in IEEE Transactions on Neural Systems and Rehabilitation Engineering.


Refbacks

  • There are currently no refbacks.