Deploying machine learning (ML) models in the wild is challenging as it suffers from distribution shifts, where the model trained on an original domain cannot generalize well to unforeseen diverse transfer domains. To address this challenge, several test-time adaptation (TTA) methods have been proposed to improve the generalization ability of the target pre-trained models under test data to cope with the shifted distribution. The success of TTA can be credited to the continuous fine-tuning of the target model according to the distributional hint from the test samples during test time. Despite being powerful, it also opens a new attack surface, i.e., test-time poisoning attacks, which are substantially different from previous poisoning attacks that occur during the training time of ML models (i.e., adversaries cannot intervene in the training process). In this paper, we perform the first test-time poisoning attack against four mainstream TTA methods, including TTT, DUA, TENT, and RPL. Concretely, we generate poisoned samples based on the surrogate models and feed them to the target TTA models. Experimental results show that the TTA methods are generally vulnerable to test-time poisoning attacks. For instance, the adversary can feed as few as 10 poisoned samples to degrade the performance of the target model from 76.20% to 41.83%. Our results demonstrate that TTA algorithms lacking a rigorous security assessment are unsuitable for deployment in real-life scenarios. As such, we advocate for the integration of defenses against test-time poisoning attacks into the design of TTA methods.

翻译：在现实环境中部署机器学习模型具有挑战性，因其面临分布偏移问题——即模型在原始域上训练后无法很好地泛化至未知的多样迁移域。针对这一难题，研究者提出了多种测试时自适应方法，通过利用测试数据的分布线索在测试阶段持续微调目标预训练模型，以提升其泛化能力并应对偏移分布。尽管测试时自适应技术表现出强大性能，但它也暴露了新的攻击面：测试时投毒攻击。这与传统发生在模型训练阶段的投毒攻击（即攻击者无法干预训练过程）存在本质差异。本文首次针对TTT、DUA、TENT和RPL四种主流测试时自适应方法实施测试时投毒攻击。具体而言，我们基于替代模型生成投毒样本，并将其注入目标测试时自适应模型。实验结果表明，测试时自适应方法普遍易受此类攻击：例如，攻击者仅需投喂10个投毒样本即可使目标模型性能从76.20%骤降至41.83%。我们的研究证实，缺乏严格安全评估的测试时自适应算法不适合部署于真实场景。为此，我们主张在测试时自适应方法的设计中集成针对测试时投毒攻击的防御机制。