Generating intermediate steps, or Chain of Thought (CoT), is an effective way to significantly improve language models' (LM) multi-step reasoning capability. However, the CoT lengths can grow rapidly with the problem complexity, easily exceeding the maximum context size. Instead of increasing the context limit, which has already been heavily investigated, we explore an orthogonal direction: making LMs divide a problem into multiple contexts. We propose a new inference framework, called Recursion of Thought (RoT), which introduces several special tokens that the models can output to trigger context-related operations. Extensive experiments with multiple architectures including GPT-3 show that RoT dramatically improves LMs' inference capability to solve problems, whose solution consists of hundreds of thousands of tokens.

翻译：生成中间步骤（即思维链，Chain of Thought, CoT）是显著提升语言模型多步推理能力的有效方法。然而，随着问题复杂度的增加，思维链长度会快速增长，极易超出最大上下文限制。针对这一挑战，我们探索了一个正交方向：让语言模型将问题分解到多个上下文中进行处理，而非沿用已被广泛研究的上下文长度扩展方法。为此，我们提出了名为"思想递归"（Recursion of Thought, RoT）的新型推理框架，该框架引入若干特殊标记，使模型能够通过输出这些标记来触发与上下文相关的操作。在包括GPT-3在内的多种架构上开展的大量实验表明，RoT能够大幅提升语言模型对需要数十万标记才能完成推理的问题的解决能力。