Distribution shifts between training and test data are inevitable over the lifecycle of a deployed model, leading to performance decay. Adapting a model on test samples can help mitigate this drop in performance. However, most test-time adaptation methods have focused on synthetic corruption shifts, leaving a variety of distribution shifts underexplored. In this paper, we focus on distribution shifts that evolve gradually over time, which are common in the wild but challenging for existing methods, as we show. To address this, we propose STAD, a probabilistic state-space model that adapts a deployed model to temporal distribution shifts by learning the time-varying dynamics in the last set of hidden features. Without requiring labels, our model infers time-evolving class prototypes that act as a dynamic classification head. Through experiments on real-world temporal distribution shifts, we show that our method excels in handling small batch sizes and label shift.

翻译：在模型部署的生命周期中，训练数据与测试数据之间的分布偏移是不可避免的，这会导致模型性能下降。在测试样本上对模型进行适应有助于缓解这种性能下降。然而，大多数测试时适应方法主要关注合成损坏引起的分布偏移，而对各种其他类型的分布偏移探索不足。在本文中，我们关注随时间逐渐演变的分布偏移，这类偏移在现实场景中十分常见，但正如我们所展示的，对现有方法而言具有挑战性。为解决这一问题，我们提出了STAD，一种概率状态空间模型，它通过学习最后一组隐藏特征中随时间变化的动态特性，使已部署的模型能够适应时序分布偏移。我们的模型无需标签即可推断出随时间演化的类别原型，这些原型充当动态分类头。通过在真实世界的时序分布偏移上进行实验，我们证明了我们的方法在处理小批量数据和标签偏移方面表现出色。