Test-time adaptation (TTA) enhances the zero-shot robustness under distribution shifts by leveraging unlabeled test data during inference. Despite notable advances, several challenges still limit its broader applicability. First, most methods rely on backpropagation or iterative optimization, which limits scalability and hinders real-time deployment. Second, they lack explicit modeling of class-conditional feature distributions. This modeling is crucial for producing reliable decision boundaries and calibrated predictions, but it remains underexplored due to the lack of both source data and supervision at test time. In this paper, we propose ADAPT, an Advanced Distribution-Aware and backPropagation-free Test-time adaptation method. We reframe TTA as a Gaussian probabilistic inference task by modeling class-conditional likelihoods using gradually updated class means and a shared covariance matrix. This enables closed-form, training-free inference. To correct potential likelihood bias, we introduce lightweight regularization guided by CLIP priors and a historical knowledge bank. ADAPT requires no source data, no gradient updates, and no full access to target data, supporting both online and transductive settings. Extensive experiments across diverse benchmarks demonstrate that our method achieves state-of-the-art performance under a wide range of distribution shifts with superior scalability and robustness.

翻译：测试时自适应（TTA）通过在推理过程中利用未标注的测试数据，增强了模型在分布偏移下的零样本鲁棒性。尽管取得了显著进展，若干挑战仍限制了其更广泛的应用。首先，大多数方法依赖反向传播或迭代优化，这限制了可扩展性并阻碍了实时部署。其次，这些方法缺乏对类条件特征分布的显式建模。此类建模对于生成可靠的决策边界和校准的预测至关重要，但由于测试时既无源数据也无监督信号，该方向仍未被充分探索。本文提出ADAPT，一种先进的分布感知且无需反向传播的测试时自适应方法。我们通过使用逐步更新的类均值与共享协方差矩阵对类条件似然进行建模，将TTA重新构建为高斯概率推断任务。这使得无需训练、仅需闭式解即可完成推断。为纠正潜在的似然偏差，我们引入了基于CLIP先验和历史知识库的轻量级正则化。ADAPT无需源数据、无需梯度更新、无需完整访问目标数据，同时支持在线与直推式设置。在多种基准测试上的大量实验表明，我们的方法在广泛的分布偏移下实现了最先进的性能，并具有卓越的可扩展性和鲁棒性。