We present READ Avatars, a 3D-based approach for generating 2D avatars that are driven by audio input with direct and granular control over the emotion. Previous methods are unable to achieve realistic animation due to the many-to-many nature of audio to expression mappings. We alleviate this issue by introducing an adversarial loss in the audio-to-expression generation process. This removes the smoothing effect of regression-based models and helps to improve the realism and expressiveness of the generated avatars. We note furthermore, that audio should be directly utilized when generating mouth interiors and that other 3D-based methods do not attempt this. We address this with audio-conditioned neural textures, which are resolution-independent. To evaluate the performance of our method, we perform quantitative and qualitative experiments, including a user study. We also propose a new metric for comparing how well an actor's emotion is reconstructed in the generated avatar. Our results show that our approach outperforms state of the art audio-driven avatar generation methods across several metrics. A demo video can be found at \url{https://youtu.be/QSyMl3vV0pA}

翻译：我们提出READ Avatars，一种基于三维方法生成由音频输入驱动的二维虚拟形象的技术，能够直接且精细地控制情感表达。由于音频到表情映射的多对多特性，现有方法无法实现逼真的动画。我们通过在音频到表情生成过程中引入对抗性损失来缓解这一问题。这消除了基于回归模型的平滑效应，有助于提升生成虚拟形象的逼真度和表现力。此外，我们注意到，在生成嘴部内部细节时应直接利用音频，而其他基于三维的方法并未尝试这一点。我们通过音频条件神经纹理（分辨率无关）解决了这一问题。为评估我们方法的性能，我们进行了定量和定性实验，包括一项用户研究。我们还提出了一种新指标，用于比较生成虚拟形象中演员情感重建的效果。结果表明，我们的方法在多个指标上优于最先进的音频驱动虚拟形象生成方法。演示视频可在 \url{https://youtu.be/QSyMl3vV0pA} 查看。