Recently, large language models (LLMs) have gained much attention for the emergence of human-comparable capabilities and huge potential. However, for open-domain implicit question-answering problems, LLMs may not be the ultimate solution due to the reasons of: 1) uncovered or out-of-date domain knowledge, 2) one-shot generation and hence restricted comprehensiveness. To this end, this work proposes a gradual knowledge excavation framework for open-domain complex question answering, where LLMs iteratively and actively acquire external information, and then reason based on acquired historical knowledge. Specifically, during each step of the solving process, the model selects an action to execute, such as querying external knowledge or performing a single logical reasoning step, to gradually progress toward a final answer. Our method can effectively leverage plug-and-play external knowledge and dynamically adjust the strategy for solving complex questions. Evaluated on the StrategyQA dataset, our method achieves 78.17% accuracy with less than 6% parameters of its competitors, setting new SOTA for ~10B-scale LLMs.

翻译：近年来，大型语言模型（LLMs）因其展现出类人能力与巨大潜力而备受关注。然而，对于开放域隐式问答问题，LLMs可能并非终极解决方案，原因在于：1）未覆盖或过时的领域知识；2）一次性生成模式导致答案全面性受限。为此，本文提出一种面向开放域复杂问答的渐进式知识挖掘框架，使LLMs能够迭代式主动获取外部信息，并基于已获取的历史知识进行推理。具体而言，在求解过程的每一步，模型选择执行一个动作（例如查询外部知识或执行单步逻辑推理），从而逐步推进至最终答案。我们的方法能有效利用即插即用式外部知识，并动态调整复杂问题的求解策略。在StrategyQA数据集上的评估表明，该方法以不足竞争对手6%的参数量实现了78.17%的准确率，为约100亿参数规模的LLMs树立了新的性能标杆。