We propose a method for adapting neural networks to distribution shifts at test-time. In contrast to training-time robustness mechanisms that attempt to anticipate and counter the shift, we create a closed-loop system and make use of a test-time feedback signal to adapt a network on the fly. We show that this loop can be effectively implemented using a learning-based function, which realizes an amortized optimizer for the network. This leads to an adaptation method, named Rapid Network Adaptation (RNA), that is notably more flexible and orders of magnitude faster than the baselines. Through a broad set of experiments using various adaptation signals and target tasks, we study the efficiency and flexibility of this method. We perform the evaluations using various datasets (Taskonomy, Replica, ScanNet, Hypersim, COCO, ImageNet), tasks (depth, optical flow, semantic segmentation, classification), and distribution shifts (Cross-datasets, 2D and 3D Common Corruptions) with promising results. We end with a discussion on general formulations for handling distribution shifts and our observations from comparing with similar approaches from other domains.

翻译：我们提出了一种在测试时适应分布偏移的神经网络方法。与试图预测并应对偏移的训练时鲁棒性机制不同，我们构建了一个闭环系统，利用测试时的反馈信号动态调整网络。研究表明，该闭环可通过一个基于学习的函数高效实现，该函数充当网络的摊销优化器。由此衍生的适应方法——快速网络适应（RNA）——比基线方法更具灵活性，且速度提升数个数量级。通过一系列使用不同适应信号和目标任务的广泛实验，我们研究了该方法的效率与灵活性。我们采用多种数据集（Taskonomy、Replica、ScanNet、Hypersim、COCO、ImageNet）、任务（深度估计、光流、语义分割、分类）和分布偏移场景（跨数据集、二维及三维常见扰动）进行评估，结果令人满意。最后，我们讨论了处理分布偏移的通用框架，并分享了与其他领域类似方法对比得出的观察结论。