Domain shift is a common problem in the realistic world, where training data and test data follow different data distributions. To deal with this problem, fully test-time adaptation (TTA) leverages the unlabeled data encountered during test time to adapt the model. In particular, Entropy-Based TTA (EBTTA) methods, which minimize the prediction's entropy on test samples, have shown great success. In this paper, we introduce a new perspective on the EBTTA, which interprets these methods from a view of clustering. It is an iterative algorithm: 1) in the assignment step, the forward process of the EBTTA models is the assignment of labels for these test samples, and 2) in the updating step, the backward process is the update of the model via the assigned samples. Based on the interpretation, we can gain a deeper understanding of EBTTA, where we show that the entropy loss would further increase the largest probability. Accordingly, we offer an alternative explanation that why existing EBTTA methods are sensitive to initial assignments, outliers, and batch size. This observation can guide us to put forward the improvement of EBTTA. We propose robust label assignment, weight adjustment, and gradient accumulation to alleviate the above problems. Experimental results demonstrate that our method can achieve consistent improvements on various datasets. Code is provided in the supplementary material.
翻译:域偏移是现实世界中常见的问题,即训练数据与测试数据遵循不同的数据分布。为应对此问题,全测试时自适应方法利用测试阶段遇到的未标注数据来调整模型。其中,基于熵的测试时自适应方法通过最小化测试样本预测的熵值取得了显著成功。本文从聚类视角引入对EBTTA方法的新解读,将其视为迭代算法:1)在分配步骤中,EBTTA模型的前向过程为测试样本分配标签;2)在更新步骤中,反向过程通过已分配样本更新模型。基于这一解读,我们可更深入理解EBTTA,揭示熵损失会进一步增大最大概率值。据此提出对现有EBTTA方法敏感于初始分配、异常值与批大小的替代性解释。该观察指导我们提出改进方案,包括鲁棒标签分配、权重调整与梯度累积以缓解上述问题。实验结果表明,我们的方法在多种数据集上均能实现一致的性能提升。代码见补充材料。