Current Virtual Reality systems are designed for interaction under visual control. Using built-in cameras, headsets track the user's hands or hand-held controllers while they are inside the field of view. Current systems thus ignore the user's interaction with off-screen content -- virtual objects that the user could quickly access through proprioception without requiring laborious head motions to bring them into focus. In this paper, we present HOOV, a wrist-worn sensing method that allows VR users to interact with objects outside their field of view. Based on the signals of a single wrist-worn inertial sensor, HOOV continuously estimates the user's hand position in 3-space to complement the headset's tracking as the hands leave the tracking range. Our novel data-driven method predicts hand positions and trajectories from just the continuous estimation of hand orientation, which by itself is stable based solely on inertial observations. Our inertial sensing simultaneously detects finger pinching to register off-screen selection events, confirms them using a haptic actuator inside our wrist device, and thus allows users to select, grab, and drop virtual content. We compared HOOV's performance with a camera-based optical motion capture system in two folds. In the first evaluation, participants interacted based on tracking information from the motion capture system to assess the accuracy of their proprioceptive input, whereas in the second, they interacted based on HOOV's real-time estimations. We found that HOOV's target-agnostic estimations had a mean tracking error of 7.7 cm, which allowed participants to reliably access virtual objects around their body without first bringing them into focus. We demonstrate several applications that leverage the larger input space HOOV opens up for quick proprioceptive interaction, and conclude by discussing the potential of our technique.

翻译：当前虚拟现实系统旨在视觉控制下进行交互。利用内置摄像头，头戴设备可在用户双手位于视野内时追踪其手部或手持控制器。因此，现有系统忽略了用户与屏幕外内容的交互——即用户可通过本体感觉快速触及而无需费力转动头部使其聚焦的虚拟对象。本文提出HOOV，一种腕戴式感知方法，允许VR用户与视野外的物体进行交互。基于单个腕戴惯性传感器的信号，HOOV持续估计用户手部在三维空间中的位置，以在双手离开追踪范围时补充头戴设备的追踪能力。我们新颖的数据驱动方法仅通过连续估计手部朝向（该朝向本身基于纯惯性观测即能保持稳定）来预测手部位置和轨迹。同步地，我们的惯性感知可检测手指捏合以注册屏幕外选择事件，并通过腕戴设备内置的触觉致动器予以确认，从而让用户能够选择、抓取和释放虚拟内容。我们从两个方面将HOOV的性能与基于摄像头的光学运动捕捉系统进行了比较。在第一项评估中，参与者依据运动捕捉系统的追踪信息进行交互，以评估其本体感觉输入的准确性；而在第二项评估中，参与者依据HOOV的实时估计进行交互。我们发现，HOOV的目标无关估计平均追踪误差为7.7厘米，这使得参与者能够可靠地访问身体周围的虚拟对象，而无需先使其聚焦。我们展示了利用HOOV打开的更大输入空间实现快速本体感觉交互的若干应用，并最后讨论了该技术的潜力。