Recent progress in neural rendering has brought forth pioneering methods, such as NeRF and Gaussian Splatting, which revolutionize view rendering across various domains like AR/VR, gaming, and content creation. While these methods excel at interpolating {\em within the training data}, the challenge of generalizing to new scenes and objects from very sparse views persists. Specifically, modeling 3D humans from sparse views presents formidable hurdles due to the inherent complexity of human geometry, resulting in inaccurate reconstructions of geometry and textures. To tackle this challenge, this paper leverages recent advancements in Gaussian Splatting and introduces a new method to learn generalizable human Gaussians that allows photorealistic and accurate view-rendering of a new human subject from a limited set of sparse views in a feed-forward manner. A pivotal innovation of our approach involves reformulating the learning of 3D Gaussian parameters into a regression process defined on the 2D UV space of a human template, which allows leveraging the strong geometry prior and the advantages of 2D convolutions. In addition, a multi-scaffold is proposed to effectively represent the offset details. Our method outperforms recent methods on both within-dataset generalization as well as cross-dataset generalization settings.

翻译：神经渲染领域的最新进展催生了诸如NeRF和高斯泼溅等开创性方法，这些方法彻底改变了增强现实/虚拟现实、游戏和内容创作等多个领域的视角渲染。尽管这些方法在训练数据内部插值方面表现出色，但从极稀疏视角泛化到新场景和物体仍面临持续挑战。具体而言，由于人体几何固有的复杂性，从稀疏视角建模三维人体存在巨大障碍，导致几何与纹理重建不准确。为应对这一挑战，本文利用高斯泼溅技术的最新进展，提出了一种学习可泛化人体高斯模型的新方法，能够以前馈方式从有限稀疏视角集对新人体对象进行逼真且精确的视角渲染。我们方法的核心创新在于将三维高斯参数的学习重新定义为人体模板二维UV空间上的回归过程，从而充分利用强几何先验和二维卷积的优势。此外，本文提出多支架结构以有效表征偏移细节。我们的方法在数据集内泛化与跨数据集泛化场景中均优于现有最新方法。