We study allowing large language models (LLMs) to process arbitrarily long prompts through the lens of inference-time scaling. We propose Recursive Language Models (RLMs), a general inference strategy that treats long prompts as part of an external environment and allows the LLM to programmatically examine, decompose, and recursively call itself over snippets of the prompt. We find that RLMs successfully handle inputs up to two orders of magnitude beyond model context windows and, even for shorter prompts, dramatically outperform the quality of base LLMs and common long-context scaffolds across four diverse long-context tasks, while having comparable (or cheaper) cost per query.

翻译：本研究从推理时扩展的视角出发，探索使大型语言模型（LLMs）能够处理任意长提示的方法。我们提出递归语言模型（RLMs），这是一种通用的推理策略，将长提示视为外部环境的一部分，允许LLM以编程方式检查、分解提示片段，并递归调用自身处理这些片段。我们发现，RLMs能够成功处理超出模型上下文窗口两个数量级的输入，并且即使在较短的提示上，在四个不同的长上下文任务中，其性能也显著优于基础LLMs和常见的长上下文框架，同时每次查询的成本相当（或更低）。