With the prevalence of the Pretraining-Finetuning paradigm in transfer learning, the robustness of downstream tasks has become a critical concern. In this work, we delve into adversarial robustness in transfer learning and reveal the critical role of initialization, including both the pretrained model and the linear head. First, we discover the necessity of an adversarially robust pretrained model. Specifically, we reveal that with a standard pretrained model, Parameter-Efficient Finetuning (PEFT) methods either fail to be adversarially robust or continue to exhibit significantly degraded adversarial robustness on downstream tasks, even with adversarial training during finetuning. Leveraging a robust pretrained model, surprisingly, we observe that a simple linear probing can outperform full finetuning and other PEFT methods with random initialization on certain datasets. We further identify that linear probing excels in preserving robustness from the robust pretraining. Based on this, we propose Robust Linear Initialization (RoLI) for adversarial finetuning, which initializes the linear head with the weights obtained by adversarial linear probing to maximally inherit the robustness from pretraining. Across five different image classification datasets, we demonstrate the effectiveness of RoLI and achieve new state-of-the-art results. Our code is available at \url{https://github.com/DongXzz/RoLI}.

翻译：随着预训练-微调范式在迁移学习中的普及，下游任务的鲁棒性已成为关键关注点。本研究深入探讨了迁移学习中的对抗鲁棒性，揭示了初始化（包括预训练模型和线性分类头）的关键作用。首先，我们发现对抗鲁棒预训练模型的必要性——具体而言，若使用标准预训练模型，参数高效微调（PEFT）方法即使采用对抗训练进行微调，仍无法实现对抗鲁棒性或在下游任务中表现出显著降低的对抗鲁棒性。令人惊讶的是，基于鲁棒预训练模型，我们观察到简单线性探针（linear probing）在某些数据集上可超越全微调及随机初始化的其他PEFT方法。进一步研究发现，线性探针在保留预训练鲁棒性方面表现优异。基于此，我们提出对抗性微调的鲁棒线性初始化（RoLI）方法，该方法通过对抗线性探针获得的权重初始化线性分类头，以最大程度继承预训练模型的鲁棒性。在五个图像分类数据集上的实验表明，RoLI方法取得有效性并达到新的最先进水平。我们的代码开源发布于 \url{https://github.com/DongXzz/RoLI}。