Recently, a range of neural network-based methods for image rendering have been introduced. One such widely-researched neural radiance field (NeRF) relies on a neural network to represent 3D scenes, allowing for realistic view synthesis from a small number of 2D images. However, most NeRF models are constrained by long training and inference times. In comparison, Gaussian Splatting (GS) is a novel, state-of-the-art technique for rendering points in a 3D scene by approximating their contribution to image pixels through Gaussian distributions, warranting fast training and swift, real-time rendering. A drawback of GS is the absence of a well-defined approach for its conditioning due to the necessity to condition several hundred thousand Gaussian components. To solve this, we introduce the Gaussian Mesh Splatting (GaMeS) model, which allows modification of Gaussian components in a similar way as meshes. We parameterize each Gaussian component by the vertices of the mesh face. Furthermore, our model needs mesh initialization on input or estimated mesh during training. We also define Gaussian splats solely based on their location on the mesh, allowing for automatic adjustments in position, scale, and rotation during animation. As a result, we obtain a real-time rendering of editable GS.

翻译：近期，一系列基于神经网络的图像渲染方法被提出。其中，广泛研究的神经辐射场（NeRF）依赖神经网络表示3D场景，能够从少量2D图像中合成逼真视图。然而，多数NeRF模型受限于较长的训练和推理时间。相比之下，高斯泼溅（GS）作为一种新颖的先进技术，通过高斯分布近似3D场景点对图像像素的贡献，实现了快速训练与实时渲染。GS的一个缺陷在于其条件化缺乏明确定义的方法——由于需要对数十万个高斯分量进行条件化处理。为此，我们提出高斯网格泼溅（GaMeS）模型，该模型支持以类似网格的方式修改高斯分量。我们将每个高斯分量参数化为网格面的顶点。此外，我们的模型需要在输入时初始化网格，或在训练过程中估计网格。我们还完全基于高斯泼溅在网格上的位置对其进行定义，从而在动画过程中实现位置、尺度和旋转的自动调整。最终，我们实现了可编辑GS的实时渲染。