Users often ask dialogue systems ambiguous questions that require clarification. We show that current language models rarely ask users to clarify ambiguous questions and instead provide incorrect answers. To address this, we introduce CLAM: a framework for getting language models to selectively ask for clarification about ambiguous user questions. In particular, we show that we can prompt language models to detect whether a given question is ambiguous, generate an appropriate clarifying question to ask the user, and give a final answer after receiving clarification. We also show that we can simulate users by providing language models with privileged information. This lets us automatically evaluate multi-turn clarification dialogues. Finally, CLAM significantly improves language models' accuracy on mixed ambiguous and unambiguous questions relative to SotA.

翻译：摘要：用户向对话系统提出的问题常存在歧义，需要系统进行澄清。本研究发现，现有语言模型很少主动要求用户澄清歧义问题，而是直接给出错误答案。为解决这一问题，我们提出了CLAM框架：一种引导语言模型对用户歧义问题选择性发起澄清提问的方法。具体而言，我们证明可以通过提示技术使语言模型：检测给定问题是否存在歧义，生成合适的澄清问题向用户提问，并在获得澄清后给出最终答案。我们还展示了如何通过向语言模型提供特权信息来模拟用户行为，从而实现对多轮澄清对话的自动评估。最后，与当前最优方法相比，CLAM显著提升了语言模型在混合歧义与非歧义问题场景中的回答准确率。