Large Language Models (LLMs) that can continually improve beyond their training budgets are able to solve increasingly difficult problems by adapting at test time, a property we refer to as extrapolation. However, standard reinforcement learning (RL) operates over fixed problem distributions and training budgets, which limits extrapolation amidst distribution shift at test time. To address this, we introduce RC, an iterative decoding algorithm that replaces standard autoregressive decoding during both training and inference. RC exploits an asymmetry between the response generation and summarization capabilities of LLMs to construct reasoning chains that consistently improve across iterations. Models trained to use RC can extrapolate and continually improve over reasoning horizons more than an order of magnitude longer than those seen during training. Empirically, training a 4B model with RC using a 16k-token training budget improves performance on HMMT 2025 from 40% to nearly 70% with 0.5m tokens at test time, outperforming both comparably sized models and many larger reasoning LLMs. Finally, we also show that models trained with RC can more effectively leverage existing scaffolds to further scale test-time performance, due to the improved summary-conditioned generation abilities learned through training.

翻译：能够超越其训练预算持续改进的大型语言模型（LLM）可通过在测试时进行自适应来解决日益复杂的问题，我们将此特性称为外推能力。然而，标准强化学习（RL）在固定的问题分布和训练预算下运行，这限制了其在测试时面对分布偏移时的外推能力。为解决这一问题，我们提出RC——一种在训练和推理阶段均替代标准自回归解码的迭代解码算法。RC利用LLM在响应生成与摘要归纳能力间的不对称性，构建在迭代过程中持续改进的推理链。经训练使用RC的模型能够实现外推，并在推理视距上持续改进，其视距长度可超过训练所见视距一个数量级以上。实证表明，使用16k词元训练预算对4B参数模型进行RC训练后，在测试时使用0.5百万词元可将HMMT 2025任务上的性能从40%提升至近70%，其表现优于同等规模模型及许多更大的推理型LLM。最后，我们还证明经RC训练的模型能更有效地利用现有推理框架进一步提升测试时性能，这得益于训练过程中习得的改进型摘要条件生成能力。