Recent advances in natural language processing, primarily propelled by Large Language Models (LLMs), have showcased their remarkable capabilities grounded in in-context learning. A promising avenue for guiding LLMs in intricate reasoning tasks involves the utilization of intermediate reasoning steps within the Chain-of-Thought (CoT) paradigm. Nevertheless, the central challenge lies in the effective selection of exemplars for facilitating in-context learning. In this study, we introduce a framework that leverages Dual Queries and Low-rank approximation Re-ranking (DQ-LoRe) to automatically select exemplars for in-context learning. Dual Queries first query LLM to obtain LLM-generated knowledge such as CoT, then query the retriever to obtain the final exemplars via both question and the knowledge. Moreover, for the second query, LoRe employs dimensionality reduction techniques to refine exemplar selection, ensuring close alignment with the input question's knowledge. Through extensive experiments, we demonstrate that DQ-LoRe significantly outperforms prior state-of-the-art methods in the automatic selection of exemplars for GPT-4, enhancing performance from 92.5% to 94.2%. Our comprehensive analysis further reveals that DQ-LoRe consistently outperforms retrieval-based approaches in terms of both performance and adaptability, especially in scenarios characterized by distribution shifts. DQ-LoRe pushes the boundary of in-context learning and opens up new avenues for addressing complex reasoning challenges. Our code is released at https://github.com/AI4fun/DQ-LoRe}{https://github.com/AI4fun/DQ-LoRe.

翻译：近期自然语言处理的进展主要得益于大型语言模型（LLMs）的推动，这些模型在上下文学习中展现了卓越能力。为引导LLMs处理复杂推理任务，链式思维（Chain-of-Thought, CoT）范式中的中间推理步骤提供了重要途径。然而，核心挑战在于如何有效选择示例以促进上下文学习。本研究提出一种基于双重查询与低秩近似重排序（DQ-LoRe）的框架，可自动选择上下文学习示例。双重查询首先向LLM查询以获取其生成的CoT等知识，随后基于问题与知识二次查询检索器以获取最终示例。针对二次查询，LoRe采用降维技术优化示例选择，确保与输入问题知识的高度对齐。通过大量实验证明，DQ-LoRe在GPT-4的自动示例选择任务中显著超越先前最优方法，性能从92.5%提升至94.2%。综合分析进一步表明，DQ-LoRe在性能与适应性方面始终优于基于检索的方法，尤其在分布偏移场景中表现突出。DQ-LoRe拓展了上下文学习的边界，为应对复杂推理挑战开辟了新路径。我们的代码已发布在https://github.com/AI4fun/DQ-LoRe。