Continual test-time adaptation (CTTA) has recently emerged to adapt a pre-trained source model to continuously evolving target distributions, which accommodates the dynamic nature of real-world environments. To mitigate the risk of catastrophic forgetting in CTTA, existing methods typically incorporate explicit regularization terms to constrain the variation of model parameters. However, they cannot fundamentally resolve catastrophic forgetting because they rely on a single shared model to adapt across all target domains, which inevitably leads to severe inter-domain interference. In this paper, we introduce learnable domain-specific prompts that guide the model to adapt to corresponding target domains, thereby partially disentangling the parameter space of different domains. In the absence of domain identity for target samples, we propose a novel dynamic Prompt AllocatIon aNd Tuning (PAINT) method, which utilizes a query mechanism to dynamically determine whether the current samples come from a known domain or an unexplored one. For known domains, the corresponding domain-specific prompt is directly selected, while for previously unseen domains, a new prompt is allocated. Prompt tuning is subsequently performed using mutual information maximization along with structural regularization. Extensive experiments on three benchmark datasets demonstrate the effectiveness of our PAINT method for CTTA. We have released our code at https://github.com/Cadezzyr/PAINT.
翻译:持续测试时适应(CTTA)近年来兴起,旨在将预训练的源模型适应持续变化的目标分布,以适应现实环境的动态特性。为减轻CTTA中的灾难性遗忘风险,现有方法通常引入显式正则化项以约束模型参数的变化。然而,这些方法无法从根本上解决灾难性遗忘问题,因为它们依赖单一共享模型来适应所有目标域,这不可避免地导致严重的域间干扰。本文引入可学习的域特定提示,引导模型适应相应的目标域,从而部分解耦不同域的参空间。在目标样本缺乏域身份信息的情况下,我们提出一种新颖的动态提示分配与调优(PAINT)方法,该方法利用查询机制动态判定当前样本来自已知域还是未探索域。对于已知域,直接选择对应的域特定提示;对于先前未见域,则分配新提示。随后通过互信息最大化与结构正则化进行提示调优。在三个基准数据集上的大量实验证明了我们PAINT方法在CTTA中的有效性。代码已发布于https://github.com/Cadezzyr/PAINT。