We present 3D Points Splatting Hand Reconstruction (3D-PSHR), a real-time and photo-realistic hand reconstruction approach. We propose a self-adaptive canonical points upsampling strategy to achieve high-resolution hand geometry representation. This is followed by a self-adaptive deformation that deforms the hand from the canonical space to the target pose, adapting to the dynamic changing of canonical points which, in contrast to the common practice of subdividing the MANO model, offers greater flexibility and results in improved geometry fitting. To model texture, we disentangle the appearance color into the intrinsic albedo and pose-aware shading, which are learned through a Context-Attention module. Moreover, our approach allows the geometric and the appearance models to be trained simultaneously in an end-to-end manner. We demonstrate that our method is capable of producing animatable, photorealistic and relightable hand reconstructions using multiple datasets, including monocular videos captured with handheld smartphones and large-scale multi-view videos featuring various hand poses. We also demonstrate that our approach achieves real-time rendering speeds while simultaneously maintaining superior performance compared to existing state-of-the-art methods.

翻译：我们提出三维点喷溅手部重建方法（3D-PSHR），一种实时且照片级真实感的手部重建技术。我们采用自适应规范点升采样策略，实现高分辨率手部几何表示。随后通过自适应形变将手部从规范空间变形至目标姿态，该过程适应规范点的动态变化——与常规的MANO模型细分方法不同，本方案具有更高灵活性并优化了几何拟合效果。在纹理建模方面，我们将外观颜色解耦为固有反照率和姿态感知明暗分量，并通过上下文注意力模块学习其特征。此外，本方法支持几何模型与外观模型以端到端方式进行同步训练。实验表明，该方法可利用多数据集（包括手持智能手机采集的单目视频及包含多种手部姿态的大规模多视角视频）生成可驱动、照片级真实感且可重光照的手部重建结果。同时，相比现有最优方法，我们的方案在保持卓越性能的同时实现了实时渲染速度。