Open-ended question answering requires models to find appropriate evidence to form well-reasoned, comprehensive and helpful answers. In practical applications, models also need to engage in extended discussions on potential scenarios closely relevant to the question. With augmentation of retrieval module, open-source Large Language Models (LLMs) can produce coherent answers often with different focuses, but are still sub-optimal in terms of reliable evidence selection and in-depth question analysis. In this paper, we propose a novel Chain-of-Discussion framework to leverage the synergy among multiple open-source LLMs aiming to provide \textbf{more correct} and \textbf{more comprehensive} answers for open-ended QA, although they are not strong enough individually. Our experiments show that discussions among multiple LLMs play a vital role in enhancing the quality of answers. We release our data and code at \url{https://github.com/kobayashikanna01/Chain-of-Discussion}.

翻译：开放式问答要求模型找到适当的证据，以形成推理充分、全面且有用的答案。在实际应用中，模型还需对与问题密切相关的潜在场景展开深入讨论。借助检索模块的增强，开源大型语言模型（LLMs）能生成连贯但常侧重不同维度的答案，然而在可靠证据选择和深度问题分析方面仍存在不足。本文提出一种新颖的"讨论链"框架，通过利用多个开源LLMs（尽管单个能力不够强）的协同作用，旨在为开放式问答提供\textbf{更正确}且\textbf{更全面}的答案。实验表明，多LLM之间的讨论对提升答案质量至关重要。我们已在\url{https://github.com/kobayashikanna01/Chain-of-Discussion}上公开了数据和代码。