HoloMoCap: Real-Time Clinical Motion Capture with HoloLens 2

Abstract

Low-cost, portable motion capture (MoCap) systems struggle to achieve the same accuracy as the marker-based gold standard, and often fail to provide real-time feedback on patients' motion parameters. To address these challenges, we present HoloMoCap, a novel marker-based MoCap system enabling clinicians to track and visualize human movements in real-time through a head-mounted Augmented Reality (AR) display. HoloMoCap is a HoloLens 2 stand-alone application, requiring no external tracking systems or additional servers for motion analysis. The application utilizes the HoloLens' depth sensor, operating at 5 frames per second (fps), to perform inside-out tracking of infrared markers attached to the patient's skin. At each frame, the system detects reflective markers, uses an on-device Deep Learning (DL) model to associate each marker with its corresponding body landmark, and calculates anatomical joint angles (hip and knee flexion, abduction, and rotation). Validation against Vicon was performed during rehabilitation exercises (squats and hip abduction), showing that estimated joint angles maintain root-mean-square error (RMSE) and mean absolute error (MAE) below 2° for most angles. HoloMoCap accurately estimated the range of motion (ROM) for hip abduction and knee flexion, with average MAEs of 0.4° and 1.2°, respectively. However, for hip flexion, the MAE can exceed 10° at maximum flexion during squats. HoloMoCap shows promise as a portable and cost-effective solution for motion capture, although further improvements in accuracy and frame rate are necessary to broaden its clinical applications.