Transformer-Based Estimation of Pilot Cognitive Workload Using Self-Supervised Multimodal Biometric Learning

Abstract

Accurately estimating a pilot’s cognitive workload is essential for enhancing aviation safety and fostering the development of intelligent cockpit systems. Conventional machine learning models, though effective in narrowly defined domains, often struggle to generalize across diverse flight conditions. This paper presents a comparative study of two workload classification approaches. They are: transfer learning using the convolutional BM3TX architecture and a Transformer-based model inspired by the Joint Embedding Predictive Architecture (JEPA). Using biometric data collected from 89 pilots performing simulated turbulence maneuvers, our results demonstrate that the JEPA-inspired model significantly outperforms the traditional approach in both accuracy (70.6% vs. 63.8%) and interpretability. Attention-based visualizations provide additional insight into the decision-making process, offering a foundation for scalable, explainable pilot monitoring systems.