Reconstructing high-fidelity 3D head avatars is crucial in various applications such as virtual reality. The pioneering methods reconstruct realistic head avatars with Neural Radiance Fields (NeRF), which have been limited by training and rendering speed. Recent methods based on 3D Gaussian Splatting (3DGS) significantly improve the efficiency of training and rendering. However, the surface inconsistency of 3DGS results in subpar geometric accuracy; later, 2DGS uses 2D surfels to enhance geometric accuracy at the expense of rendering fidelity. To leverage the benefits of both 2DGS and 3DGS, we propose a novel method named MixedGaussianAvatar for realistically and geometrically accurate head avatar reconstruction. Our main idea is to utilize 2D Gaussians to reconstruct the surface of the 3D head, ensuring geometric accuracy. We attach the 2D Gaussians to the triangular mesh of the FLAME model and connect additional 3D Gaussians to those 2D Gaussians where the rendering quality of 2DGS is inadequate, creating a mixed 2D-3D Gaussian representation. These 2D-3D Gaussians can then be animated using FLAME parameters. We further introduce a progressive training strategy that first trains the 2D Gaussians and then fine-tunes the mixed 2D-3D Gaussians. We demonstrate the superiority of MixedGaussianAvatar through comprehensive experiments. The code will be released at: https://github.com/ChenVoid/MGA/.
翻译:高保真三维头部化身的重建在虚拟现实等众多应用中至关重要。开创性方法利用神经辐射场(NeRF)重建具有真实感的头部化身,但其训练与渲染速度一直受限。近期基于三维高斯溅射(3DGS)的方法显著提升了训练与渲染效率。然而,3DGS的表面不一致性导致其几何精度欠佳;随后提出的2DGS采用二维面元来提升几何精度,但牺牲了渲染保真度。为了兼收2DGS与3DGS的优势,我们提出了一种名为MixedGaussianAvatar的新方法,用于实现兼具真实感与几何精度的头部化身重建。我们的核心思路是利用二维高斯分布重建三维头部的表面,以确保几何精度。我们将这些二维高斯分布附着于FLAME模型的三角网格上,并在2DGS渲染质量不足的区域,为这些二维高斯连接额外的三维高斯分布,从而构建出混合的2D-3D高斯表征。这些2D-3D高斯分布随后可通过FLAME参数进行驱动动画。我们进一步提出了一种渐进式训练策略:先训练二维高斯分布,再对混合的2D-3D高斯分布进行微调。通过全面的实验,我们验证了MixedGaussianAvatar的优越性。代码将发布于:https://github.com/ChenVoid/MGA/。