One of the most challenging problems in audio-driven talking head generation is achieving high-fidelity detail while ensuring precise synchronization. Given only a single reference image, extracting meaningful identity attributes becomes even more challenging, often causing the network to mirror the facial and lip structures too closely. To address these issues, we introduce RADIO, a framework engineered to yield high-quality dubbed videos regardless of the pose or expression in reference images. The key is to modulate the decoder layers using latent space composed of audio and reference features. Additionally, we incorporate ViT blocks into the decoder to emphasize high-fidelity details, especially in the lip region. Our experimental results demonstrate that RADIO displays high synchronization without the loss of fidelity. Especially in harsh scenarios where the reference frame deviates significantly from the ground truth, our method outperforms state-of-the-art methods, highlighting its robustness. Pre-trained model and codes will be made public after the review.

翻译：音频驱动的说话人脸生成中最具挑战性的问题之一是在确保精确同步的同时实现高保真细节。仅凭单张参考图像，提取有意义的身份属性变得更加困难，常常导致网络过度模仿面部和唇部结构。为解决这些问题，我们提出了RADIO，一个旨在无论参考图像姿态或表情如何，都能生成高质量配音视频的框架。其关键在于利用音频和参考特征组成的潜在空间来调制解码器层。此外，我们在解码器中融入ViT模块以强调高保真细节，尤其是唇部区域。实验结果表明，RADIO在保持高保真度的同时展现出优异同步性能。尤其是在参考帧与真实情况存在显著偏差的严苛场景中，我们的方法优于现有最先进方法，凸显了其鲁棒性。预训练模型与代码将在评审后公开。