We introduce GaussianSwap, a novel video face swapping framework that constructs a 3D Gaussian Splatting based face avatar from a target video while transferring identity from a source image to the avatar. Conventional video swapping frameworks are limited to generating facial representations in pixel-based formats. The resulting swapped faces exist merely as a set of unstructured pixels without any capacity for animation or interactive manipulation. Our work introduces a paradigm shift from conventional pixel-based video generation to the creation of high-fidelity avatar with swapped faces. The framework first preprocesses target video to extract FLAME parameters, camera poses and segmentation masks, and then rigs 3D Gaussian splats to the FLAME model across frames, enabling dynamic facial control. To ensure identity preserving, we propose an compound identity embedding constructed from three state-of-the-art face recognition models for avatar finetuning. Finally, we render the face-swapped avatar on the background frames to obtain the face-swapped video. Experimental results demonstrate that GaussianSwap achieves superior identity preservation, visual clarity and temporal consistency, while enabling previously unattainable interactive applications.

翻译：本文提出GaussianSwap，一种新颖的视频人脸交换框架，该框架从目标视频构建基于3D高斯溅射的人脸化身，同时将源图像的身份特征迁移至该化身。传统视频交换框架仅限于生成基于像素格式的面部表征，所得交换人脸仅作为一组非结构化像素存在，不具备任何动画或交互操控能力。本研究实现了从传统基于像素的视频生成到高保真度人脸交换化身创建的范式转变。该框架首先对目标视频进行预处理以提取FLAME参数、相机位姿与分割掩码，随后将3D高斯溅射点绑定至跨帧的FLAME模型，从而实现动态面部控制。为确保身份特征保持，我们提出一种复合身份嵌入方法，该方法通过三种前沿人脸识别模型构建，用于化身的微调优化。最后，我们将人脸交换后的化身渲染至背景帧，以获取人脸交换视频。实验结果表明，GaussianSwap在身份保持度、视觉清晰度与时间一致性方面均表现优异，同时实现了以往无法达成的交互式应用场景。