Recent advances in natural language processing, primarily propelled by Large Language Models (LLMs), have showcased their remarkable capabilities grounded in in-context learning. A promising avenue for guiding LLMs in intricate reasoning tasks involves the utilization of intermediate reasoning steps within the Chain-of-Thought (CoT) paradigm. Nevertheless, the central challenge lies in the effective selection of exemplars for facilitating in-context learning. In this study, we introduce a framework that leverages Dual Queries and Low-rank approximation Re-ranking (DQ-LoRe) to automatically select exemplars for in-context learning. Dual Queries first query LLM to obtain LLM-generated knowledge such as CoT, then query the retriever to obtain the final exemplars via both question and the knowledge. Moreover, for the second query, LoRe employs dimensionality reduction techniques to refine exemplar selection, ensuring close alignment with the input question's knowledge. Through extensive experiments, we demonstrate that DQ-LoRe significantly outperforms prior state-of-the-art methods in the automatic selection of exemplars for GPT-4, enhancing performance from 92.5% to 94.2%. Our comprehensive analysis further reveals that DQ-LoRe consistently outperforms retrieval-based approaches in terms of both performance and adaptability, especially in scenarios characterized by distribution shifts. DQ-LoRe pushes the boundary of in-context learning and opens up new avenues for addressing complex reasoning challenges. Our code is released at https://github.com/menik1126/DQ-LoRe

翻译：近年来，自然语言处理领域——主要得益于大语言模型（LLMs）的推动——在上下文学习方面展现出卓越的能力。在思维链（CoT）范式中，利用中间推理步骤来引导LLMs完成复杂推理任务，已成为一种前景广阔的研究方向。然而，如何有效选择用于促进上下文学习的范例仍是核心挑战。本研究提出了一个利用双查询与低秩近似重排序（DQ-LoRe）的框架，以自动选择上下文学习所需的范例。双查询首先查询LLM以获取LLM生成的知识（如CoT），随后结合问题与知识再次查询检索器，以获取最终范例。此外，在第二次查询中，LoRe采用降维技术来优化范例选择，确保其与输入问题所蕴含的知识高度契合。通过大量实验，我们证明在GPT-4的自动范例选择任务中，DQ-LoRe显著优于以往最先进的方法，将性能从92.5%提升至94.2%。我们的综合分析进一步表明，DQ-LoRe在性能和适应性方面均持续优于基于检索的方法，尤其在存在分布偏移的场景中表现突出。DQ-LoRe推动了上下文学习的边界，并为应对复杂推理挑战开辟了新途径。代码已发布于 https://github.com/menik1126/DQ-LoRe。