Real-world image recognition systems often face corrupted input images, which cause distribution shifts and degrade the performance of models. These systems often use a single prediction model in a central server and process images sent from various environments, such as cameras distributed in cities or cars. Such single models face images corrupted in heterogeneous ways in test time. Thus, they require to instantly adapt to the multiple corruptions during testing rather than being re-trained at a high cost. Test-time adaptation (TTA), which aims to adapt models without accessing the training dataset, is one of the settings that can address this problem. Existing TTA methods indeed work well on a single corruption. However, the adaptation ability is limited when multiple types of corruption occur, which is more realistic. We hypothesize this is because the distribution shift is more complicated, and the adaptation becomes more difficult in case of multiple corruptions. In fact, we experimentally found that a larger distribution gap remains after TTA. To address the distribution gap during testing, we propose a novel TTA method named Covariance-Aware Feature alignment (CAFe). We empirically show that CAFe outperforms prior TTA methods on image corruptions, including multiple types of corruptions.

翻译：真实世界的图像识别系统常面临输入图像损坏的情况，这会导致分布偏移并降低模型性能。此类系统通常使用中央服务器中的单一预测模型，处理来自不同环境（如分布于城市或车辆中的摄像头）发送的图像。这种单一模型在测试阶段会面临以异质方式损坏的图像，因此需要即时适应多种损坏类型，而非以高成本重新训练。测试时自适应（TTA）旨在无需访问训练数据集的情况下适配模型，是解决该问题的典型方案之一。现有TTA方法在单一损坏类型上表现良好，但在更符合实际的多类型损坏发生时，其自适应能力受限。我们推测这是因为分布偏移更复杂，且多重损坏情形下自适应难度更高。实际上，实验发现TTA后仍存在较大的分布间隙。针对测试阶段的分布间隙问题，我们提出名为协方差感知特征对齐（CAFe）的新型TTA方法。经验证，CAFe在图像损坏（包括多类型损坏）场景下的性能优于现有TTA方法。