We tackle the challenging task of generating complete 3D facial animations for two interacting, co-located participants from a mixed audio stream. While existing methods often produce disembodied "talking heads" akin to a video conference call, our work is the first to explicitly model the dynamic 3D spatial relationship -- including relative position, orientation, and mutual gaze -- that is crucial for realistic in-person dialogues. Our system synthesizes the full performance of both individuals, including precise lip-sync, and uniquely allows their relative head poses to be controlled via textual descriptions. To achieve this, we propose a dual-stream architecture where each stream is responsible for one participant's output. We employ speaker's role embeddings and inter-speaker cross-attention mechanisms designed to disentangle the mixed audio and model the interaction. Furthermore, we introduce a novel eye gaze loss to promote natural, mutual eye contact. To power our data-hungry approach, we introduce a novel pipeline to curate a large-scale conversational dataset consisting of over 2 million dyadic pairs from in-the-wild videos. Our method generates fluid, controllable, and spatially aware dyadic animations suitable for immersive applications in VR and telepresence, significantly outperforming existing baselines in perceived realism and interaction coherence.

翻译：我们解决了从混合音频流中为两个交互的、共置的参与者生成完整三维面部动画的挑战性任务。现有方法通常产生类似视频会议中脱离身体的"说话头部"，而我们的工作首次明确建模了动态的三维空间关系——包括相对位置、朝向和相互注视——这对于逼真的面对面对话至关重要。我们的系统合成了两个个体的完整表现，包括精确的唇形同步，并独特地允许通过文本描述控制他们的相对头部姿态。为实现这一目标，我们提出了一种双流架构，其中每个流负责一个参与者的输出。我们采用说话者角色嵌入和说话者间交叉注意力机制，旨在解耦混合音频并建模交互。此外，我们引入了一种新颖的视线损失函数，以促进自然、相互的眼神接触。为支持我们数据密集型的方法，我们引入了一种新颖的流水线来策划一个大规模对话数据集，该数据集包含来自真实世界视频的超过200万对二元对话对。我们的方法生成流畅、可控且具有空间意识的二元动画，适用于虚拟现实和远程呈现中的沉浸式应用，在感知真实性和交互连贯性方面显著优于现有基线。