We have recently seen tremendous progress in photo-real human modeling and rendering. Yet, efficiently rendering realistic human performance and integrating it into the rasterization pipeline remains challenging. In this paper, we present HiFi4G, an explicit and compact Gaussian-based approach for high-fidelity human performance rendering from dense footage. Our core intuition is to marry the 3D Gaussian representation with non-rigid tracking, achieving a compact and compression-friendly representation. We first propose a dual-graph mechanism to obtain motion priors, with a coarse deformation graph for effective initialization and a fine-grained Gaussian graph to enforce subsequent constraints. Then, we utilize a 4D Gaussian optimization scheme with adaptive spatial-temporal regularizers to effectively balance the non-rigid prior and Gaussian updating. We also present a companion compression scheme with residual compensation for immersive experiences on various platforms. It achieves a substantial compression rate of approximately 25 times, with less than 2MB of storage per frame. Extensive experiments demonstrate the effectiveness of our approach, which significantly outperforms existing approaches in terms of optimization speed, rendering quality, and storage overhead.

翻译：我们近期目睹了照片级真实感人体建模与渲染的巨大进展。然而，高效渲染逼真的人体表现并将其集成到光栅化流水线中仍具挑战性。本文提出HiFi4G，一种基于显式紧凑高斯表示的高保真人体表现渲染方法，可处理密集多视角影像。其核心思路是将三维高斯表示与非刚性追踪相结合，实现紧凑且利于压缩的表示。我们首先提出双图机制获取运动先验：粗变形图用于有效初始化，细粒度高斯图用于强化后续约束。随后采用带自适应时空正则化的四维高斯优化方案，有效平衡非刚性先验与高斯更新。我们还提出配套的残差补偿压缩方案，可在多种平台上实现沉浸式体验。该方案实现约25倍的高压缩率，每帧存储量低于2MB。大量实验证明，本方法在优化速度、渲染质量和存储开销上显著超越现有方法。