The feed-forward based 3D Gaussian Splatting method has demonstrated exceptional capability in real-time human novel view synthesis. However, existing approaches are restricted to dense viewpoint settings, which limits their flexibility in free-viewpoint rendering across a wide range of camera view angle discrepancies. To address this limitation, we propose a real-time pipeline named EVA-Gaussian for 3D human novel view synthesis across diverse camera settings. Specifically, we first introduce an Efficient cross-View Attention (EVA) module to accurately estimate the position of each 3D Gaussian from the source images. Then, we integrate the source images with the estimated Gaussian position map to predict the attributes and feature embeddings of the 3D Gaussians. Moreover, we employ a recurrent feature refiner to correct artifacts caused by geometric errors in position estimation and enhance visual fidelity.To further improve synthesis quality, we incorporate a powerful anchor loss function for both 3D Gaussian attributes and human face landmarks. Experimental results on the THuman2.0 and THumansit datasets showcase the superiority of our EVA-Gaussian approach in rendering quality across diverse camera settings. Project page: https://zhenliuzju.github.io/huyingdong/EVA-Gaussian.
翻译:基于前馈的3D高斯溅射方法在实时人体新视角合成中展现出卓越能力。然而,现有方法受限于密集视点设置,这限制了其在广泛相机视角差异范围内进行自由视点渲染的灵活性。为解决这一局限,我们提出了一种名为EVA-Gaussian的实时处理流程,用于多相机设置下的3D人体新视角合成。具体而言,我们首先引入高效的跨视角注意力(EVA)模块,以从源图像中精确估计每个3D高斯分布的位置。随后,我们将源图像与估计的高斯位置图相结合,以预测3D高斯分布的属性与特征嵌入。此外,我们采用循环特征优化器来修正位置估计中几何误差导致的伪影,并提升视觉保真度。为进一步提升合成质量,我们针对3D高斯属性与人脸关键点设计了强效的锚点损失函数。在THuman2.0与THumansit数据集上的实验结果表明,我们的EVA-Gaussian方法在多相机设置下的渲染质量具有显著优势。项目页面:https://zhenliuzju.github.io/huyingdong/EVA-Gaussian。