As large language models (LLMs) increasingly permeate daily lives, there is a growing demand for real-time interactions that mirror human conversations. Traditional turn-based chat systems driven by LLMs prevent users from verbally interacting with the system while it is generating responses. To overcome these limitations, we adapt existing LLMs to \textit{duplex models} so that these LLMs can listen for users while generating output and dynamically adjust themselves to provide users with instant feedback. % such as in response to interruptions. Specifically, we divide the queries and responses of conversations into several time slices and then adopt a time-division-multiplexing (TDM) encoding-decoding strategy to pseudo-simultaneously process these slices. Furthermore, to make LLMs proficient enough to handle real-time conversations, we build a fine-tuning dataset consisting of alternating time slices of queries and responses as well as covering typical feedback types in instantaneous interactions. Our experiments show that although the queries and responses of conversations are segmented into incomplete slices for processing, LLMs can preserve their original performance on standard benchmarks with a few fine-tuning steps on our dataset. Automatic and human evaluation indicate that duplex models make user-AI interactions more natural and human-like, and greatly improve user satisfaction compared to vanilla LLMs. Our duplex model and dataset will be released.

翻译：随着大语言模型（LLMs）日益渗透日常生活，人们对能够模拟人类对话的实时交互的需求日益增长。传统的、由LLMs驱动的回合制聊天系统在生成回复时会阻止用户与系统进行语音交互。为克服这些限制，我们将现有LLMs适配为\textit{双工模型}，使得这些LLMs能够在生成输出的同时监听用户输入，并动态调整自身以向用户提供即时反馈。具体而言，我们将对话中的查询和响应划分为多个时间片，随后采用时分复用（TDM）编码-解码策略对这些时间片进行伪并行处理。此外，为使LLMs足够熟练地处理实时对话，我们构建了一个微调数据集，其中包含交替排列的查询与响应时间片，并涵盖了即时交互中的典型反馈类型。实验表明，尽管对话的查询和响应被分割为不完整的片段进行处理，LLMs仅需在我们的数据集上进行少量微调步骤，即可在标准基准测试中保持其原始性能。自动评估与人工评估均表明，与原始LLMs相比，双工模型使用户与AI的交互更加自然、拟人化，并显著提升了用户满意度。我们的双工模型与数据集将予以公开。