Fusion of physiological signals for modeling driver awareness levels in conditional autonomous vehicles using semi-supervised learning

Fernandez Matellan, RaulPuertas Ramírez, DavidMartín Gómez, DavidGonzález Boticario, Jesús2025-09-152025-09-152024-10-11R. Fernandez-Matellan, D. Puertas-Ramirez, D. M. Gomez and J. G. Boticario, "Fusion of Physiological Signals for Modeling Driver Awareness Levels in Conditional Autonomous Vehicles using Semi-Supervised Learning," 2024 27th International Conference on Information Fusion (FUSION), Venice, Italy, 2024, pp. 1-8, doi: 10.23919/FUSION59988.2024.107065179781737749769https://doi.org/10.23919/FUSION59988.2024.10706517https://hdl.handle.net/20.500.14468/30067The registered version of this conference paper, first published in "2024 27th International Conference on Information Fusion (FUSION), Venice, Italy, 2024, pp. 1-8", is available online at the publisher's website: https://doi.org/10.23919/FUSION59988.2024.10706517The evolution of autonomous vehicles (AVs) requires a paradigm shift towards the integration of human factors to improve safety and efficiency at levels 2,3 and 4 of automation. This paper presents a comparison of three different fusion technologies (Low-Level fusion, Medium-Level fusion, and a hybrid fusion), highlighting the critical role of multimodal data integration and semi-supervised learning in predicting and adapting to levels of driver awareness. Our approach uses semi-supervised learning to deal with the data labelling problem, using unlabelled data to train an autoencoder and sparsely labelled data to train a 4-state classifier. Our model facilitates the fusion of data from different physiological signals, including skin electrodermal activity, heart rate, body temperature and acceleration. Using real driving data, the Medium-Level fusion approach gives the best performance, achieving 84% accuracy in predicting situations where the user may not be aware enough to take control of the vehicle. This research highlights the essential nature of fusion technologies to create adaptive and user-centred AV systems.eninfo:eu-repo/semantics/closedAccess3304 Tecnología de los ordenadoresFusion of physiological signals for modeling driver awareness levels in conditional autonomous vehicles using semi-supervised learningactas de congreso