Test-time adaptation (TTA) aims to adapt a trained classifier using online unlabeled test data only, without any information related to the training procedure. Most existing TTA methods adapt the trained classifier using the classifier's prediction on the test data as pseudo-label. However, under test-time domain shift, accuracy of the pseudo labels cannot be guaranteed, and thus the TTA methods often encounter performance degradation at the adapted classifier. To overcome this limitation, we propose a novel test-time adaptation method, called Test-time Adaptation via Self-Training with nearest neighbor information (TAST), which is composed of the following procedures: (1) adds trainable adaptation modules on top of the trained feature extractor; (2) newly defines a pseudo-label distribution for the test data by using the nearest neighbor information; (3) trains these modules only a few times during test time to match the nearest neighbor-based pseudo label distribution and a prototype-based class distribution for the test data; and (4) predicts the label of test data using the average predicted class distribution from these modules. The pseudo-label generation is based on the basic intuition that a test data and its nearest neighbor in the embedding space are likely to share the same label under the domain shift. By utilizing multiple randomly initialized adaptation modules, TAST extracts useful information for the classification of the test data under the domain shift, using the nearest neighbor information. TAST showed better performance than the state-of-the-art TTA methods on two standard benchmark tasks, domain generalization, namely VLCS, PACS, OfficeHome, and TerraIncognita, and image corruption, particularly CIFAR-10/100C.

翻译：测试时自适应（TTA）旨在仅利用在线无标签测试数据，在不依赖任何训练过程相关信息的情况下，对训练好的分类器进行自适应调整。现有大多数TTA方法利用分类器对测试数据的预测结果作为伪标签来调整模型。然而，在测试时域偏移下，伪标签的准确性无法得到保证，因此TTA方法常导致自适应后的分类器性能下降。为解决这一局限，我们提出了一种新颖的测试时自适应方法，称为基于最近邻信息的自训练测试时自适应（TAST），其包含以下步骤：（1）在训练好的特征提取器之上添加可训练的自适应模块；（2）利用最近邻信息为测试数据新定义伪标签分布；（3）在测试期间仅对模块进行少量训练，使基于最近邻的伪标签分布与基于原型的测试数据类分布相匹配；（4）利用这些模块的预测类分布均值来预测测试数据的标签。伪标签生成基于一个基本直觉：在域偏移下，测试数据与其在嵌入空间中的最近邻很可能共享同一标签。通过利用多个随机初始化的自适应模块，TAST借助最近邻信息提取对域偏移下测试数据分类有用的信息。在域泛化的两个标准基准任务（VLCS、PACS、OfficeHome和TerraIncognita）以及图像损坏任务（特别是CIFAR-10/100C）上，TAST均展现出优于现有最优TTA方法的性能。