Mixed reality applications can be designed for hand rehabilitation. Augmented reality (AR) head mounted displays (HMDs) specifically allow for ecologically valid tasks because individuals can see their real environment and interact with real objects while receiving additional cues on the HMD. While these applications rely on accurate hand pose estimation, there is a gap in investigating the influence of hand impairment or occlusion from real-object interactions on pose estimation accuracy. Further, comparisons between AR HMD predictions and state-of-the-art pose estimation methods have not been established. The current study assessed pose estimation accuracy of the HoloLens 2 HMD and state-of-the-art pose estimation algorithms (WiLoR, HaMeR, WildHands, and MediaPipe) while individuals with cervical spinal cord injury (cSCI; n = 13, Neurological Level of Injury: C3-C6; American Spinal Injury Association Impairment Scale: A-D) and 15 uninjured controls interacted with clear and opaque objects. Ground truth estimates of 3D joint positions were generated via triangulation from a multi-camera setup. Pose estimation accuracy did not differ between the cSCI and uninjured control groups suggesting that 3D joint predictions from the HoloLens 2 and pose estimation algorithms can generalize to populations with hand impairment. Further, clear objects provided a small accuracy advantage over opaque objects (0.1 mm) and predictions from both WiLoR and HaMeR were slightly more accurate than the HoloLens 2 (2 mm). Overall, these results suggest that the HoloLens 2 may be viable for hand rehabilitation applications and the dataset generated can be used to refine pose estimation methods for hand-impaired populations.

翻译：混合现实应用可设计用于手部康复训练。增强现实头戴显示设备通过允许用户在保持真实环境视觉的同时与真实物体交互，并接收设备叠加的辅助提示，特别适用于生态效度任务。尽管此类应用依赖于精准的手部姿态估计，但关于手部损伤或真实物体交互产生的遮挡对姿态估计精度的影响仍存在研究空白。此外，增强现实头戴显示设备的预测结果与最新姿态估计方法之间的系统性比较尚未建立。本研究评估了HoloLens 2头戴显示设备及最新姿态估计算法（WiLoR、HaMeR、WildHands和MediaPipe）在颈椎脊髓损伤患者（n=13，神经损伤平面：C3-C6；美国脊髓损伤协会损伤分级：A-D）与15名无损伤对照组参与者与透明及不透明物体交互时的姿态估计精度。通过多相机系统三角测量获得三维关节点位置的真值。结果显示，cSCI组与无损伤对照组之间的姿态估计精度无显著差异，表明HoloLens 2与姿态估计算法的三维关节点预测可推广至手部损伤人群。透明物体相较于不透明物体具有微小精度优势（0.1毫米），而WiLoR与HaMeR的预测精度略优于HoloLens 2（2毫米）。总体而言，这些结果表明HoloLens 2可能适用于手部康复应用，且生成的标注数据集可用于优化针对手部损伤人群的姿态估计方法。