Large Language Models (LLMs) has shown exceptional capabilities in many natual language understanding and generation tasks. However, the personalization issue still remains a much-coveted property, especially when it comes to the multiple sources involved in the dialogue system. To better plan and incorporate the use of multiple sources in generating personalized response, we firstly decompose it into three sub-tasks: Knowledge Source Selection, Knowledge Retrieval, and Response Generation. We then propose a novel Unified Multi-Source Retrieval-Augmented Generation system (UniMS-RAG) Specifically, we unify these three sub-tasks with different formulations into the same sequence-to-sequence paradigm during the training, to adaptively retrieve evidences and evaluate the relevance on-demand using special tokens, called acting tokens and evaluation tokens. Enabling language models to generate acting tokens facilitates interaction with various knowledge sources, allowing them to adapt their behavior to diverse task requirements. Meanwhile, evaluation tokens gauge the relevance score between the dialogue context and the retrieved evidence. In addition, we carefully design a self-refinement mechanism to iteratively refine the generated response considering 1) the consistency scores between the generated response and retrieved evidence; and 2) the relevance scores. Experiments on two personalized datasets (DuLeMon and KBP) show that UniMS-RAG achieves state-of-the-art performance on the knowledge source selection and response generation task with itself as a retriever in a unified manner. Extensive analyses and discussions are provided for shedding some new perspectives for personalized dialogue systems.

翻译：大型语言模型（LLMs）在诸多自然语言理解与生成任务中展现出卓越能力。然而，个性化问题仍是其亟需突破的关键属性，尤其在对话系统涉及多源信息时更为突出。为更好地规划并整合多源信息以生成个性化响应，我们首先将该任务分解为三个子任务：知识源选择、知识检索和响应生成。进而提出一种新型统一多源检索增强生成系统（UniMS-RAG）。具体而言，我们在训练过程中将这三个具有不同形式的子任务统一至相同的序列到序列范式中，利用称为动作令牌（acting tokens）和评估令牌（evaluation tokens）的特殊令牌，自适应地按需检索证据并评估相关性。使语言模型能够生成动作令牌，便于其与各类知识源交互，从而根据多样化任务需求调整行为。同时，评估令牌用于衡量对话上下文与检索证据之间的相关性得分。此外，我们精心设计了一种自我精炼机制，综合考虑（1）生成响应与检索证据之间的一致性得分，以及（2）相关性得分，对生成响应进行迭代优化。在两个个性化数据集（DuLeMon与KBP）上的实验表明，UniMS-RAG以统一方式集检索器于自身，在知识源选择与响应生成任务上达到了最先进性能。我们提供了广泛的分析与讨论，旨在为个性化对话系统开辟新视角。