Physiology-driven cybersickness detection in virtual reality: a machine learning and explainable AI approach

Abstract

One of the major obstacles to the widespread adoption of Virtual Reality (VR) is Cybersickness. It is a sense of physical discomfort akin to motion sickness experienced by the users either during or subsequent to VR utilization. Typically, it is detected through explicit methods such as self-reported questionnaires, which are not ideal for online and implicit monitoring of users' well-being. This study tackles the challenge of implicitly detecting and measuring Cybersickness in VR through physiological signals. Therefore, we propose a multimodal approach that integrates physiological signals with self-reported measures. We utilize a mixed-methods design combining quantitative and qualitative analyses, using a roller coaster simulation as the experimental paradigm. The research analyzes physiological data collected from a group of 22 participants exposed to a simulated VR rollercoaster experience, through statistical analysis and machine learning algorithms. The physiological markers studied include Electroencephalograms (EEG), Electrodermal Activity, Blood Volume Pulse, and skin Temperature signals. Additionally, the study elucidates the complex interplay among physiological markers using Explainable AI (XAI). We found out significant correlations between high Simulator Sickness Questionnaire scores and physiological measures such as brain rhythms and EEG indices related to engagement, visual fatigue, and drowsiness, as well as heart rate variability. Moreover, the proposed machine learning model achieves an accuracy of 86.66 % in detecting elevated Cybersickness symptoms, which is higher than the accuracy achieved by existing techniques. Further exploration using the XAI technique confirms the elevated levels of drowsiness and reduced engagement in participants experiencing elevated Cybersickness. By providing a comprehensive, multimodal approach for quantitative assessment, this study fills a gap in the existing literature and paves the path for the development of adaptive systems that modulate their behavior based on physiological data.