Physiology-Driven User Perception Prediction for Networked Collaborative Virtual Reality

Abstract

The growing need for remote immersive collaboration across various societal fields like education, healthcare, and training has given rise to the emergence of Networked Collaborative Virtual Reality (NCVR), where users interact in shared virtual spaces over the network. In such context, the effectiveness of the collaboration will be determined by the quality of the interaction among the users and their perceived quality. However, understanding the user perception in NCVR is complex due to the interplay between network conditions, especially latency and individual user responses. Thus, there is a need for accurate modeling of user perception that could provide fast assessments of the user perception during the collaborative session. Traditionally, assessing user perception has been performed either by subjective studies, where users rank the perceived quality of the content after the session, or by objective metrics, which provide a measure of the quality of the content as compared to the original counterpart. These approaches fall short with NCVR, where not only the quality of the content, but of the interaction and the user’s well-being will be key. Physiological signals offer a promising pathway to capture the nuances of the user experience, yet their analysis is challenging due to their inherent complexity and individual variability. This paper makes two key contributions: first, it presents a thorough correlation analysis between user perception and physiological signals. Second, it introduces a novel machine learning method that integrates network performance metrics with physiological data to predict user perception in NCVR environments. The proposed approach was evaluated using an experimental dataset comprising heart activity and network performance data collected from users engaged in a collaborative VR task in the presence of latency-related impairments. Results indicated that combining network data with physiological signals improves prediction accuracy, achieving up to 84% accuracy for latency perception.