Head-mounted device-based human-computer interaction often requires egocentric recognition of hand gestures and fingertips detection. In this paper, a unified approach of egocentric hand gesture recognition and fingertip detection is introduced. The proposed algorithm uses a single convolutional neural network to predict the probabilities of finger class and positions of fingertips in one forward propagation. Instead of directly regressing the positions of fingertips from the fully connected layer, the ensemble of the position of fingertips is regressed from the fully convolutional network. Subsequently, the ensemble average is taken to regress the final position of fingertips. Since the whole pipeline uses a single network, it is significantly fast in computation. Experimental results show that the proposed method outperforms the existing fingertip detection approaches including the Direct Regression and the Heatmap-based framework. The effectiveness of the proposed method is also shown in-the-wild scenario as well as in a use-case of virtual reality.

翻译：头顶设备基于人体计算机的互动往往要求以自我为中心的对手势和指尖检测的识别。在本文中,采用了以自我为中心的手势识别和指尖检测的统一方法。提议的算法使用单一的进化神经网络来预测手指阶级和指尖在前向传播中的位置的概率。建议的方法不是直接从完全相连的层直接倒退指尖的位置,而是从完全相连的网络中倒退出指尖位置的组合。随后,采用共合平均数来倒退指尖的最后位置。由于整个管道使用单一网络,因此计算速度非常快。实验结果表明,拟议的方法超过了现有的指尖检测方法,包括直接回归和以赫特马普为基础的框架。拟议的方法的有效性也表现在虚拟现实的虚幻情景中。