Despite the massive success of fine-tuning large Pre-trained Language Models (PLMs) on a wide range of Natural Language Processing (NLP) tasks, they remain susceptible to out-of-distribution (OOD) and adversarial inputs. Data map (DM) is a simple yet effective dual-model approach that enhances the robustness of fine-tuned PLMs, which involves fine-tuning a model on the original training set (i.e. reference model), selecting a specified fraction of important training examples according to the training dynamics of the reference model, and fine-tuning the same model on these selected examples (i.e. main model). However, it suffers from the drawback of requiring fine-tuning the same model twice, which is computationally expensive for large models. In this paper, we first show that 1) training dynamics are highly transferable across different model sizes and different pre-training methods, and that 2) main models fine-tuned using DM learn faster than when using conventional Empirical Risk Minimization (ERM). Building on these observations, we propose a novel fine-tuning approach based on the DM method: Fine-Tuning by transFerring Training dynamics (FTFT). Compared with DM, FTFT uses more efficient reference models and then fine-tunes more capable main models for fewer steps. Our experiments show that FTFT achieves better generalization robustness than ERM while spending less than half of the training cost.

翻译：尽管大规模预训练语言模型（PLMs）在各类自然语言处理（NLP）任务上取得了巨大成功，但它们仍然容易受到分布外（OOD）和对抗性输入的影响。数据映射（DM）是一种简单而有效的双模型方法，能够增强微调后PLMs的鲁棒性：其流程包括在原始训练集上微调一个模型（即参考模型），根据参考模型的训练动态选取特定比例的重要训练样本，然后在这些选取的样本上对同一模型进行微调（即主模型）。然而，该方法存在一个缺陷，即需要对同一模型进行两次微调，这对大型模型而言计算成本高昂。本文首先证明：1）训练动态在不同模型规模和不同预训练方法之间具有高度可迁移性；2）采用DM方法微调的主模型比使用传统经验风险最小化（ERM）方法学习得更快。基于这些发现，我们提出了一种基于DM方法的新型微调策略：通过迁移训练动态进行微调（FTFT）。与DM相比，FTFT使用更高效的参考模型，并以更少的步骤微调能力更强的主模型。实验表明，FTFT在训练开销不到一半的情况下，实现了比ERM更优的泛化鲁棒性。