Current talking face generation methods mainly focus on speech-lip synchronization. However, insufficient investigation on the facial talking style leads to a lifeless and monotonous avatar. Most previous works fail to imitate expressive styles from arbitrary video prompts and ensure the authenticity of the generated video. This paper proposes an unsupervised variational style transfer model (VAST) to vivify the neutral photo-realistic avatars. Our model consists of three key components: a style encoder that extracts facial style representations from the given video prompts; a hybrid facial expression decoder to model accurate speech-related movements; a variational style enhancer that enhances the style space to be highly expressive and meaningful. With our essential designs on facial style learning, our model is able to flexibly capture the expressive facial style from arbitrary video prompts and transfer it onto a personalized image renderer in a zero-shot manner. Experimental results demonstrate the proposed approach contributes to a more vivid talking avatar with higher authenticity and richer expressiveness.

翻译：当前说话人脸生成方法主要关注语音与嘴唇的同步。然而，对说话面部风格的探究不足导致生成的虚拟形象缺乏生机且单调。多数现有方法难以从任意视频提示中模仿富有表现力的风格，同时确保生成视频的真实性。本文提出一种无监督变分风格迁移模型(VAST)，以激活中性照片级逼真的虚拟形象。我们的模型包含三个关键组件：一个从给定视频提示中提取面部风格表示的风格编码器；一个用于建模与语音相关的精确运动的混合面部表情解码器；一个用于增强风格空间使其高度表现力且富有意义的变分风格增强器。通过对面部风格学习的关键设计，我们的模型能够灵活地从任意视频提示中捕获富有表现力的面部风格，并以零样本方式将其迁移到个性化图像渲染器上。实验结果表明，所提方法有助于生成更生动、真实感更强且表现力更丰富的说话虚拟形象。