Large Language Models (LLMs) that can continually improve beyond their training budgets are able to solve increasingly difficult problems by adapting at test time, a property we refer to as extrapolation. However, standard reinforcement learning (RL) operates over fixed problem distributions and training budgets, which limits extrapolation amidst distribution shift at test time. To address this, we introduce RC, an iterative decoding algorithm that replaces standard autoregressive decoding during both training and inference. RC exploits an asymmetry between the response generation and summarization capabilities of LLMs to construct reasoning chains that consistently improve across iterations. Models trained to use RC can extrapolate and continually improve over reasoning horizons more than an order of magnitude longer than those seen during training. Empirically, training a 4B model with RC using a 16k-token training budget improves performance on HMMT 2025 from 40% to nearly 70% with 0.5m tokens at test time, outperforming both comparably sized models and many larger reasoning LLMs. Finally, we also show that models trained with RC can more effectively leverage existing scaffolds to further scale test-time performance, due to the improved summary-conditioned generation abilities learned through training.

翻译：大型语言模型若能突破训练预算限制持续改进，则可通过测试时自适应解决日益复杂的问题，我们将此特性称为外推能力。然而，标准强化学习基于固定的问题分布和训练预算运作，这限制了模型在测试时分布偏移场景下的外推能力。为解决此问题，我们提出RC——一种在训练与推理阶段替代标准自回归解码的迭代解码算法。RC利用语言模型在响应生成与摘要生成能力间的非对称性，构建可在迭代过程中持续优化的推理链。经过RC训练的模型能外推至推理链长度超过训练时一个数量级以上的场景，并在推理链上持续改进。实验表明，使用16k令牌训练预算训练4B参数的RC模型，在HMMT 2025基准测试中通过0.5m令牌的测试时计算，性能从40%提升至近70%，不仅超越同等规模模型，还优于许多更大规模的推理语言模型。此外，我们发现经RC训练的模型由于获得了更优的摘要条件生成能力，能更有效利用现有脚手架进一步扩展测试时性能。