Test-time entropy minimization helps adapt a model to novel environments and incentivize its reasoning capability, unleashing the model's potential during inference by allowing it to evolve and improve in real-time using its own predictions, achieving promising performance. However, pure entropy minimization can favor non-generalizable shortcuts, such as inflating the logit norm and driving all predictions to a dominant class to reduce entropy, risking collapsed solutions (e.g., constant one-hot outputs) that trivially minimize the objective without meaningful learning. In this paper, we reveal asymmetry as a key mechanism for collapse prevention and introduce ZeroSiam--an efficient asymmetric Siamese architecture tailored for test-time entropy minimization. ZeroSiam prevents collapse through asymmetric divergence alignment, efficiently achieved by a learnable predictor and a stop-gradient operator before the classifier. We provide empirical and theoretical evidence that ZeroSiam not only prevents collapse, but also regularizes biased learning signals, enhancing performance even when no collapse occurs. Despite its simplicity, extensive results show that ZeroSiam performs more stably over prior methods using negligible overhead, demonstrating efficacy on both vision adaptation and large language model reasoning tasks across challenging test scenarios and diverse models, including particularly collapse-prone tiny models.

翻译：摘要：测试时熵最小化有助于使模型适应新环境并激发其推理能力，通过在推理过程中允许模型利用自身预测进行实时演进与改进，释放模型潜力，从而取得良好性能。然而，纯熵最小化可能偏向不可泛化的捷径，例如放大对数几率范数并将所有预测推向主导类别以降低熵，从而存在坍塌风险（例如产生恒定的独热输出），此类解虽能琐碎地最小化目标函数却无法实现有意义的学习。本文揭示非对称性是防止坍塌的关键机制，并提出ZeroSiam——一种专为测试时熵最小化设计的高效非对称孪生架构。ZeroSiam通过非对称散度对齐防止坍塌，该机制通过可学习预测器及分类器前的停止梯度算子高效实现。我们提供实证与理论证据表明，ZeroSiam不仅能防止坍塌，还可正则化有偏学习信号，甚至在无坍塌情况下也能提升性能。尽管其设计简洁，大量实验结果表明，ZeroSiam在仅引入可忽略开销的情况下，相比先前方法表现更稳定，在具有挑战性的测试场景及多种模型（特别是易坍塌的微型模型）上，其有效性在视觉自适应与大语言模型推理任务中均得到验证。