Human interaction is continuous, multimodal, and full-duplex by nature. Although recent omni models have made substantial progress in unified speech, vision, and text modeling, combining seamless real-time interaction with complex reasoning and tool use remains challenging. We present DuplexOmni, a method for real-time multimodal full-duplex interaction. DuplexOmni separates model capability into an interaction layer and a thinking layer, which collaborate asynchronously in parallel. The interaction layer is implemented by the DuplexOmni model, an end-to-end system that processes streaming audio and video inputs while generating text and speech responses in real time. The thinking layer is a pluggable module that provides complex reasoning and tool-use capabilities. To support this method, we further develop a Writer-Director pipeline for constructing continuous-interaction training data. Experiments show that DuplexOmni achieves strong performance on multiple public benchmarks and exhibits natural full-duplex interaction ability.

翻译：人类交互本质上具有连续性、多模态性和全双工特性。尽管近期全模态模型在语音、视觉与文本的联合建模方面取得重大进展，但在无缝实现实时交互的同时兼顾复杂推理与工具使用仍具挑战。我们提出DuplexOmni——一种实时多模态全双工交互方法。该方法将模型能力分为交互层与思考层，两层异步并行协作。其中，交互层由端到端系统DuplexOmni模型实现，可实时处理流式音频与视频输入，并同步生成文本与语音响应；思考层作为可插拔模块，提供复杂推理与工具使用能力。为支撑该方法，我们进一步开发了Writer-Director流水线以构建连续性交互训练数据。实验表明，DuplexOmni在多个公开基准测试中表现优异，并展现出自然的全双工交互能力。