Body actions and head gestures are natural interfaces for interaction in virtual environments. Existing methods for in-place body action recognition often require hardware more than a head-mounted display (HMD), making body action interfaces difficult to be introduced to ordinary virtual reality (VR) users as they usually only possess an HMD. In addition, there lacks a unified solution to recognize in-place body actions and head gestures. This potentially hinders the exploration of the use of in-place body actions and head gestures for novel interaction experiences in virtual environments. We present a unified two-stream 1-D convolutional neural network (CNN) for recognition of body actions when a user performs walking-in-place (WIP) and for recognition of head gestures when a user stands still wearing only an HMD. Compared to previous approaches, our method does not require specialized hardware and/or additional tracking devices other than an HMD and can recognize a significantly larger number of body actions and head gestures than other existing methods. In total, ten in-place body actions and eight head gestures can be recognized with the proposed method, which makes this method a readily available body action interface (head gestures included) for interaction with virtual environments. We demonstrate one utility of the interface through a virtual locomotion task. Results show that the present body action interface is reliable in detecting body actions for the VR locomotion task but is physically demanding compared to a touch controller interface. The present body action interface is promising for new VR experiences and applications, especially for VR fitness applications where workouts are intended.

翻译：身体动作与头部姿态是虚拟环境中交互的自然界面。现有原地身体动作识别方法通常需要头戴显示器（HMD）以外的硬件设备，导致此类交互界面难以推广至仅配备头戴显示器的普通虚拟现实（VR）用户。此外，目前尚缺乏能够同时识别原地身体动作与头部姿态的统一解决方案，这阻碍了将两者用于虚拟环境新型交互体验的探索。我们提出了一种统一的单流一维卷积神经网络（CNN）方法：当用户仅佩戴头戴显示器进行原地行走（WIP）时，该网络可识别其身体动作；当用户静止站立时，可识别其头部姿态。与现有方法相比，本方法除头戴显示器外无需专用硬件或额外追踪设备，且能识别的身体动作与头部姿态数量远超现有技术。该方法共可识别十种原地身体动作与八种头部姿态，从而为虚拟环境交互提供了一种即开即用的身体动作界面（含头部姿态）。通过虚拟运动任务展示了该界面的实用性：结果表明，本身体动作界面在虚拟现实运动任务中具有可靠的动作检测能力，但在体能消耗上高于触控控制器界面。该界面有望为新型虚拟现实体验与应用（特别是以锻炼为目的的虚拟现实健身应用）开辟新路径。