Portrait Animation aims to synthesize a lifelike video from a single source image, using it as an appearance reference, with motion (i.e., facial expressions and head pose) derived from a driving video, audio, text, or generation. Instead of following mainstream diffusion-based methods, we explore and extend the potential of the implicit-keypoint-based framework, which effectively balances computational efficiency and controllability. Building upon this, we develop a video-driven portrait animation framework named LivePortrait with a focus on better generalization, controllability, and efficiency for practical usage. To enhance the generation quality and generalization ability, we scale up the training data to about 69 million high-quality frames, adopt a mixed image-video training strategy, upgrade the network architecture, and design better motion transformation and optimization objectives. Additionally, we discover that compact implicit keypoints can effectively represent a kind of blendshapes and meticulously propose a stitching and two retargeting modules, which utilize a small MLP with negligible computational overhead, to enhance the controllability. Experimental results demonstrate the efficacy of our framework even compared to diffusion-based methods. The generation speed remarkably reaches 12.8ms on an RTX 4090 GPU with PyTorch. The inference code and models are available at https://github.com/KwaiVGI/LivePortrait

翻译：人像动画旨在从单张源图像合成逼真视频，将其作为外观参考，而运动（即面部表情与头部姿态）则源自驱动视频、音频、文本或生成内容。不同于主流的基于扩散模型的方法，我们探索并拓展了基于隐式关键点框架的潜力，该框架在计算效率与可控性之间实现了有效平衡。基于此，我们开发了一个名为LivePortrait的视频驱动人像动画框架，重点关注实际应用中的泛化能力、可控性与效率。为提升生成质量与泛化能力，我们将训练数据规模扩展至约6900万高质量帧，采用混合图像-视频训练策略，升级网络架构，并设计了更优的运动变换与优化目标。此外，我们发现紧凑的隐式关键点能有效表征一类混合形状，并精心提出了拼接模块与两个重定向模块——这些模块利用计算开销可忽略的小型MLP来增强可控性。实验结果表明，即使与基于扩散的方法相比，我们的框架仍展现出卓越性能。在RTX 4090 GPU与PyTorch环境下，生成速度显著达到12.8毫秒/帧。推理代码与模型已发布于https://github.com/KwaiVGI/LivePortrait