Looped transformers promise test-time compute scaling by spending more iterations on harder problems, but it remains unclear which architectural choices let them extrapolate to harder problems at test time rather than memorize training-specific solutions. We introduce a fixed-point based framework for analyzing looped architectures along three axes of stability -- reachability, input-dependence, and geometry -- and use it to characterize when fixed-point iteration yields meaningful predictions. Theoretically, we prove that looped networks without recall have countable fixed points and cannot achieve strong input-dependence at any spectral regime, while recall combined with outer normalization reliably produces a regime in which fixed points are simultaneously reachable, locally smooth in the input, and supported by stable backpropagation. Empirically, we train single-layer looped transformers on chess, sudoku, and prefix-sums and find that downstream performance tracks the framework's predictions across tasks and architectural configurations. We additionally introduce internal recall, a novel recall placement variant, and show that it becomes competitive with -- and on sudoku, substantially better than -- standard recall placement once outer normalization is applied.

翻译：环回Transformer通过在更难的测试问题上投入更多迭代次数，有望实现测试时计算缩放，但何种架构选择能让它们泛化至更难的测试问题，而非记忆训练特定解决方案仍不明确。我们引入了一个基于不动点的框架，从三个稳定性维度——可达性、输入依赖性和几何形态——分析环回架构，并利用该框架刻画了不动点迭代何时能产生有意义的预测。理论上，我们证明：缺乏记忆机制的环回网络具有可数不动点，且在任何谱域均无法实现强输入依赖性；而结合外层归一化的记忆机制能可靠地产生这样一种状态——不动点同时具备可达性、输入局部光滑性以及稳定反向传播支持性。实验上，我们在国际象棋、数独和前缀和问题上训练单层环回Transformer，发现下游性能在不同任务和架构配置下均与框架预测一致。此外，我们引入了内部记忆这一新型记忆布局变体，并证明在外层归一化应用后，它能与标准记忆布局相竞争——在数独任务上甚至显著优于后者。