Audio-driven talking head generation aims to create vivid and realistic videos from a static portrait and speech. Existing AR-based methods rely on intermediate facial representations, which limit their expressiveness and realism. Meanwhile, diffusion-based methods generate clip-by-clip, lacking fine-grained control and causing inherent latency due to overall denoising across the window. To address these limitations, we propose EARTalking, a novel end-to-end, GPT-style autoregressive model for interactive audio-driven talking head generation. Our method introduces a novel frame-by-frame, in-context, audio-driven streaming generation paradigm. For inherently supporting variable-length video generation with identity consistency, we propose the Sink Frame Window Attention (SFA) mechanism. Furthermore, to avoid the complex, separate networks that prior works required for diverse control signals, we propose a streaming Frame Condition In-Context (FCIC) scheme. This scheme efficiently injects diverse control signals in a streaming, in-context manner, enabling interactive control at every frame and at arbitrary moments. Experiments demonstrate that EARTalking outperforms existing autoregressive methods and achieves performance comparable to diffusion-based methods. Our work demonstrates the feasibility of in-context streaming autoregressive control, unlocking a scalable direction for flexible, efficient generation. The code will be released for reproducibility.

翻译：摘要：音频驱动的说话头生成旨在从静态肖像和语音中创建生动逼真的视频。现有基于自回归的方法依赖中间面部表征，这限制了其表现力和真实性。同时，基于扩散的方法逐片段生成，缺乏精细控制，且由于整体跨窗口去噪导致固有延迟。为解决这些限制，我们提出EARTalking，一种新颖的端到端、GPT风格的自回归模型，用于交互式音频驱动的说话头生成。我们的方法引入了一种新颖的逐帧、上下文感知、音频驱动的流式生成范式。为固有地支持身份一致的变长视频生成，我们提出了汇帧窗口注意力机制。此外，为避免先前工作中用于多样化控制信号的复杂独立网络，我们提出了流式帧条件上下文方案。该方案以流式、上下文方式高效注入多样化控制信号，实现对每一帧及任意时刻的交互式控制。实验表明，EARTalking优于现有自回归方法，并达到与基于扩散方法相媲美的性能。我们的工作证明了上下文流式自回归控制的可行性，为灵活高效的生成开辟了可扩展方向。代码将开源以促进可复现性。