Currently, pre-trained language models (PLMs) do not cope well with the distribution shift problem, resulting in models trained on the training set failing in real test scenarios. To address this problem, the test-time adaptation (TTA) shows great potential, which updates model parameters to suit the test data at the testing time. Existing TTA methods rely on well-designed auxiliary tasks or self-training strategies based on pseudo-label. However, these methods do not achieve good trade-offs regarding performance gains and computational costs. To obtain some insights into such a dilemma, we take two representative TTA methods, i.e., Tent and OIL, for exploration and find that stable prediction is the key to achieving a good balance. Accordingly, in this paper, we propose perturbation consistency learning (PCL), a simple test-time adaptation method to promote the model to make stable predictions for samples with distribution shifts. Extensive experiments on adversarial robustness and cross-lingual transferring demonstrate that our method can achieve higher or comparable performance with less inference time over strong PLM backbones and previous state-of-the-art TTA methods.

翻译：当前，预训练语言模型（PLMs）在处理分布偏移问题时表现不佳，导致在训练集上训练的模型在真实测试场景中失效。为解决这一问题，测试时自适应（TTA）展现出巨大潜力，该方法在测试阶段更新模型参数以适应测试数据。现有TTA方法依赖精心设计的辅助任务或基于伪标签的自训练策略。然而，这些方法在性能提升与计算成本之间未能实现良好平衡。为深入探究该困境，我们选取两种代表性TTA方法——Tent与OIL进行剖析，发现稳定预测是实现平衡的关键。据此，本文提出了一种简洁的测试时自适应方法——扰动一致性学习（PCL），旨在促进模型对分布偏移样本进行稳定预测。在对抗鲁棒性和跨语言迁移任务上的大量实验表明，相较于强PLM骨干网络及现有最先进TTA方法，我们的方法能以更少推理时间实现更高或相当的性能。