We present a unified framework for camera-space 3D hand pose estimation from a single RGB image based on 3D implicit representation. As opposed to recent works, most of which first adopt holistic or pixel-level dense regression to obtain relative 3D hand pose and then follow with complex second-stage operations for 3D global root or scale recovery, we propose a novel unified 3D dense regression scheme to estimate camera-space 3D hand pose via dense 3D point-wise voting in camera frustum. Through direct dense modeling in 3D domain inspired by Pixel-aligned Implicit Functions for 3D detailed reconstruction, our proposed Neural Voting Field (NVF) fully models 3D dense local evidence and hand global geometry, helping to alleviate common 2D-to-3D ambiguities. Specifically, for a 3D query point in camera frustum and its pixel-aligned image feature, NVF, represented by a Multi-Layer Perceptron, regresses: (i) its signed distance to the hand surface; (ii) a set of 4D offset vectors (1D voting weight and 3D directional vector to each hand joint). Following a vote-casting scheme, 4D offset vectors from near-surface points are selected to calculate the 3D hand joint coordinates by a weighted average. Experiments demonstrate that NVF outperforms existing state-of-the-art algorithms on FreiHAND dataset for camera-space 3D hand pose estimation. We also adapt NVF to the classic task of root-relative 3D hand pose estimation, for which NVF also obtains state-of-the-art results on HO3D dataset.

翻译：我们提出一个基于三维隐式表示的统一框架，用于从单张RGB图像估计相机空间中的三维手部姿态。与近期大多数方法不同——它们通常先通过整体或像素级密集回归获得相对三维手部姿态，再通过复杂的第二阶段操作进行三维全局根节点或尺度恢复——我们提出一种新颖的统一三维密集回归方案，通过在相机视锥内进行密集三维点级投票来直接估计相机空间三维手部姿态。受像素对齐隐式函数在三维精细重建中的启发，我们通过直接的三维域密集建模，提出的神经投票场(NVF)完整建模了三维密集局部证据与手部全局几何，有助于缓解常见的二维到三维歧义性。具体而言，对于相机视锥中的三维查询点及其像素对齐的图像特征，由多层感知器表示的NVF回归以下两类信息：(i)该点到手部表面的符号距离；(ii)一组四维偏移向量（包含一维投票权重和指向各手部关节的三维方向向量）。通过投票机制，选择近表面点的四维偏移向量，经加权平均计算三维手部关节坐标。实验表明，在FreiHAND数据集上，NVF在相机空间三维手部姿态估计任务中超越了现有最优算法。我们也将NVF适配到经典任务——根节点相对三维手部姿态估计中，在HO3D数据集上同样取得了最优结果。