Despite the success of integrating large language models into the development of conversational systems, many studies have shown the effectiveness of retrieving and augmenting external knowledge for informative responses. Hence, many existing studies commonly assume the always need for Retrieval Augmented Generation (RAG) in a conversational system without explicit control. This raises a research question about such a necessity. In this study, we propose to investigate the need for each turn of system response to be augmented with external knowledge. In particular, by leveraging human judgements on the binary choice of adaptive augmentation, we develop RAGate, a gating model, which models conversation context and relevant inputs to predict if a conversational system requires RAG for improved responses. We conduct extensive experiments on devising and applying RAGate to conversational models and well-rounded analyses of different conversational scenarios. Our experimental results and analysis indicate the effective application of RAGate in RAG-based conversational systems in identifying system responses for appropriate RAG with high-quality responses and a high generation confidence. This study also identifies the correlation between the generation's confidence level and the relevance of the augmented knowledge.

翻译：尽管将大型语言模型整合到对话系统开发中已取得显著成功，但多项研究表明，检索并增强外部知识对于生成信息丰富的回应具有显著效果。因此，许多现有研究通常默认对话系统始终需要检索增强生成（RAG），而缺乏显式控制机制。这引发了一个关于此种必要性的研究问题。在本研究中，我们旨在探究系统回应的每一轮次是否需要通过外部知识进行增强。具体而言，通过利用人类对自适应增强二元选择的判断，我们开发了RAGate——一个门控模型，该模型通过对对话上下文及相关输入进行建模，以预测对话系统是否需要通过RAG来提升回应质量。我们进行了大量实验，将RAGate设计与应用于对话模型，并对不同对话场景进行了全面分析。实验结果表明，RAGate在基于RAG的对话系统中能有效识别系统回应，实现恰当的RAG操作，从而获得高质量回应并保持较高的生成置信度。本研究还揭示了生成置信度与增强知识相关性之间的关联性。