We introduce training-free looped transformers, in which a lightweight inference-time wrapper loops a contiguous mid-stack block of layers of a frozen checkpoint without additional fine-tuning, continued training, or architectural changes. Unlike prior looped transformer methods that train with the looped structure end-to-end, we retrofit recurrence onto pretrained models at test time. We show that naive block reapplication usually degrades performance, highlighting the importance of the loop application strategy. Motivated by viewing a pre-norm transformer block as a forward Euler step on an ODE, we instead treat looping as a refinement of the same approximation, replacing one large update with smaller damped sub-steps. Across seven dense, sparse MoE, and MLA+MoE model families, our method improves Qwen3-4B-Instruct by +2.64 pp on MMLU-Pro, Qwen3-30B-A3B-Instruct by +1.14 pp on CommonsenseQA, and Moonlight-16B-A3B-Instruct by +1.20 pp on OpenBookQA.

翻译：我们提出免训练循环变压器，通过在推断时对冻结检查点的连续中间块层进行轻量级包装，无需额外微调、持续训练或架构更改。与以往需要以循环结构端到端训练的循环变压器方法不同，我们在测试时将循环性回溯至预训练模型。我们证明，简单的块重复通常会降低性能，凸显了循环应用策略的重要性。受将预归一化变压器块视为常微分方程上的前向欧拉步长的启发，我们将循环视为对同一近似的细化，用较小的阻尼子步骤替代一个大更新。在七个密集、稀疏MoE及MLA+MoE模型系列中，我们的方法使Qwen3-4B-Instruct在MMLU-Pro上提升+2.64个百分点，Qwen3-30B-A3B-Instruct在CommonsenseQA上提升+1.14个百分点，Moonlight-16B-A3B-Instruct在OpenBookQA上提升+1.20个百分点。