We present SplattingAvatar, a hybrid 3D representation of photorealistic human avatars with Gaussian Splatting embedded on a triangle mesh, which renders over 300 FPS on a modern GPU and 30 FPS on a mobile device. We disentangle the motion and appearance of a virtual human with explicit mesh geometry and implicit appearance modeling with Gaussian Splatting. The Gaussians are defined by barycentric coordinates and displacement on a triangle mesh as Phong surfaces. We extend lifted optimization to simultaneously optimize the parameters of the Gaussians while walking on the triangle mesh. SplattingAvatar is a hybrid representation of virtual humans where the mesh represents low-frequency motion and surface deformation, while the Gaussians take over the high-frequency geometry and detailed appearance. Unlike existing deformation methods that rely on an MLP-based linear blend skinning (LBS) field for motion, we control the rotation and translation of the Gaussians directly by mesh, which empowers its compatibility with various animation techniques, e.g., skeletal animation, blend shapes, and mesh editing. Trainable from monocular videos for both full-body and head avatars, SplattingAvatar shows state-of-the-art rendering quality across multiple datasets.

翻译：我们提出SplattingAvatar，一种将高斯泼溅嵌入三角形网格的混合式三维表示方法，用于生成逼真虚拟人化身，在现代GPU上可实现超过300 FPS的渲染速度，在移动设备上可达30 FPS。该方法通过显式网格几何与基于高斯泼溅的隐式外观建模，将虚拟人的运动与外观进行解耦。高斯函数以三角形网格上的重心坐标和位移定义，形成Phong曲面。我们扩展了提升优化方法，在网格表面移动时同步优化高斯参数。SplattingAvatar是一种混合虚拟人表示：网格负责低频运动与表面形变，而高斯函数则处理高频几何细节与精细外观。与依赖基于MLP的线性混合蒙皮（LBS）场进行运动的现有形变方法不同，我们直接通过网格控制高斯函数的旋转与平移，从而支持多种动画技术（如骨骼动画、混合形状和网格编辑）。该方法可从单目视频中训练全身与头部虚拟人，在多个数据集上均达到领先的渲染质量。