We have recently seen tremendous progress in photo-real human modeling and rendering. Yet, efficiently rendering realistic human performance and integrating it into the rasterization pipeline remains challenging. In this paper, we present HiFi4G, an explicit and compact Gaussian-based approach for high-fidelity human performance rendering from dense footage. Our core intuition is to marry the 3D Gaussian representation with non-rigid tracking, achieving a compact and compression-friendly representation. We first propose a dual-graph mechanism to obtain motion priors, with a coarse deformation graph for effective initialization and a fine-grained Gaussian graph to enforce subsequent constraints. Then, we utilize a 4D Gaussian optimization scheme with adaptive spatial-temporal regularizers to effectively balance the non-rigid prior and Gaussian updating. We also present a companion compression scheme with residual compensation for immersive experiences on various platforms. It achieves a substantial compression rate of approximately 25 times, with less than 2MB of storage per frame. Extensive experiments demonstrate the effectiveness of our approach, which significantly outperforms existing approaches in terms of optimization speed, rendering quality, and storage overhead.

翻译：我们近期在人像逼真建模与渲染方面取得了巨大进展。然而，高效渲染真实人体性能并将其集成到光栅化流水线中仍具挑战。本文提出HiFi4G，一种显式紧凑的高斯方法，用于从密集影像进行高保真人体性能渲染。核心思路是将三维高斯表示与非刚性跟踪相结合，实现紧凑且利于压缩的表示。我们首先提出双图机制获取运动先验：粗变形图用于有效初始化，细粒度高斯图用于加强后续约束。随后，采用具有自适应时空正则化器的四维高斯优化方案，有效平衡非刚性先验与高斯更新。我们还提出一种带有残差补偿的配套压缩方案，支持多平台的沉浸式体验。该方法实现了约25倍的显著压缩率，单帧存储量低于2MB。大量实验证明了该方法的有效性，在优化速度、渲染质量和存储开销方面均显著优于现有方法。