With the advance of large language models (LLMs), the research field of LLM applications becomes more and more popular and the idea of constructing pipelines to accomplish complex tasks by stacking LLM API calls come true. However, this kind of methods face two limitations: narrow information coverage and low fault tolerance. In this work, we propose a novel method called ALLIES. Given an input query, ALLIES leverages LLMs to iteratively generate new queries related to the original query, enabling an iterative reasoning process. By iteratively refining and expanding the scope of the original query, ALLIES captures and utilizes hidden knowledge that may not be directly obtainable through retrieval. We take zero-shot open-domain question answering (ODQA) as an application scene and evaluate ALLIES on the widely-used benchmarks, such as NQ, WebQ and TriviaQA. The experimental results demonstrate that ALLIES significantly outperforms other zero-shot baselines, indicating its effectiveness in tackling those challenges. Our code is available in https://github.com/microsoft/SimXNS/tree/main/ALLIES.

翻译：随着大语言模型（LLMs）的发展，LLM应用研究领域日益活跃，通过堆叠LLM API调用来构建复杂任务流程的设想已成为现实。然而，这类方法面临两大局限：信息覆盖范围狭窄和容错率低。本文提出一种名为ALLIES的新方法。给定输入查询，ALLIES利用LLM迭代生成与原查询相关的新查询，从而实现迭代推理过程。通过逐步精炼和扩展原始查询范围，ALLIES捕获并利用了检索可能无法直接获取的隐含知识。我们将零样本开放域问答（ODQA）作为应用场景，在NQ、WebQ和TriviaQA等广泛使用的基准数据集上评估ALLIES。实验结果表明，与其它零样本基线方法相比，ALLIES表现显著更优，验证了其在应对上述挑战方面的有效性。我们的代码已在https://github.com/microsoft/SimXNS/tree/main/ALLIES开源。