Test-time training (TTT) adapts a pretrained model to each prompt via parameter updates, improving accuracy under pretraining-to-test distribution shifts. Yet, its performance often suffers from instability and sensitivity to hyperparameters such as update steps and subspace. We explain this behavior through a decision-theoretic lens, treating TTT as implicit Bayesian inference in the kernel regime. Under a Gaussian process benchmark, we show that TTT reduces prediction error when updates are spectrally matched to the prompt's signal-to-noise ratio and aligned with query-relevant eigen-directions. This perspective underpins the following results: (1) we show when fixed update steps and subspaces fail under distribution shifts, motivating adaptive strategies; (2) we prove that selecting update steps via prompt evidence admits a PAC-Bayes guarantee against overfitting; and (3) we characterize the Bayes-optimal update subspace under a linear-Gaussian correction model, yielding a scoring rule for selecting Transformer blocks and heads. Our theory helps explain the empirical instability of TTT, taking a step toward principled guidance for when, how far, and which directions to adapt.

翻译：测试时训练（TTT）通过对每个提示进行参数更新来适配预训练模型，从而提高在预训练与测试分布偏移条件下的准确性。然而，其性能常受限于不稳定性和对超参数（如更新步长与子空间）的敏感性。我们通过决策理论视角解释这一行为，将TTT视为核机制下的隐式贝叶斯推断。在高斯过程基准下，我们证明：当更新步长与提示的信噪比在谱域匹配，并沿查询相关特征方向对齐时，TTT能降低预测误差。该视角支撑以下结果：（1）我们揭示了固定更新步长与子空间在分布偏移失效的原因，从而激发自适应策略；（2）我们证明通过提示证据选择更新步长，可提供过拟合风险的PAC-贝叶斯保证；（3）在线性-高斯校正模型下，我们刻画了贝叶斯最优更新子空间，进而推导出选择Transformer块与注意力头的评分规则。本理论有助于解释TTT实证的不稳定性，朝着何时、多远以及沿何方向进行适配的原则性指导迈出一步。