Test time adaptation is the process of adapting, in an unsupervised manner, a pre-trained source model to each incoming batch of the test data (i.e., without requiring a substantial portion of the test data to be available, as in traditional domain adaptation) and without access to the source data. Since it works with each batch of test data, it is well-suited for dynamic environments where decisions need to be made as the data is streaming in. Current test time adaptation methods are primarily focused on a single source model. We propose the first completely unsupervised Multi-source Test Time Adaptation (MeTA) framework that handles multiple source models and optimally combines them to adapt to the test data. MeTA has two distinguishing features. First, it efficiently obtains the optimal combination weights to combine the source models to adapt to the test data distribution. Second, it identifies which of the source model parameters to update so that only the model which is most correlated to the target data is adapted, leaving the less correlated ones untouched; this mitigates the issue of "forgetting" the source model parameters by focusing only on the source model that exhibits the strongest correlation with the test batch distribution. Experiments on diverse datasets demonstrate that the combination of multiple source models does at least as well as the best source (with hindsight knowledge), and performance does not degrade as the test data distribution changes over time (robust to forgetting).

翻译：测试时自适应是指在无监督条件下，将预训练的源模型适应于每个传入的测试数据批次（即无需像传统域适应那样需要大量测试数据可用），且不访问源数据的过程。由于该方法作用于每个测试数据批次，因此特别适用于需要实时处理流式数据的动态环境。现有的测试时自适应方法主要聚焦于单一源模型。我们提出首个完全无监督的多源测试时自适应（MeTA）框架，该框架可处理多个源模型并最优地组合它们以适应测试数据。MeTA具有两个显著特征：第一，它能高效计算最优组合权重，以组合源模型适应测试数据分布；第二，它能识别需要更新的源模型参数，仅对与目标数据相关性最高的模型进行自适应，而保持其他低相关模型不变——通过仅聚焦于与测试批次分布相关性最强的源模型，缓解了源模型参数“遗忘”问题。在多种数据集上的实验表明，多个源模型的组合效果至少不逊于最优单源模型（基于回溯性知识），且当测试数据分布随时间变化时性能不会退化（对遗忘具有鲁棒性）。