mmPrivPose3D: A dataset for pose estimation and gesture command recognition in human-robot collaboration using frequency modulated continuous wave 60Hhz RaDAR

Abstract

3D pose estimation and gesture command recognition are crucial for ensuring safety and improving human-robot interaction. While RGB-D cameras are commonly used for these tasks, they often raise privacy concerns due to their ability to capture detailed visual data of human operators. In contrast, using RaDAR sensors offers a privacy-preserving alternative, as they can output point-cloud data rather than images. We introduce mmPrivPose3D, a dataset of 3D RaDAR point-cloud data that captures human movements and gestures using a single IWR6843AOPEVM RaDAR sensor with a frequency of 10 Hz synchronized with 19 corresponding 3D skeleton keypoints as the ground truth. These keypoints were extracted from RGB-D images captured by an Intel RealSense camera recorded at 30 frames per second using the Nuitrack SDK, and labeled with gestures. The dataset was collected from n = 15 participants. Our dataset serves as a fundamental resource for developing machine learning algorithms to improve the accuracy of pose estimation and gesture recognition using RaDAR data.