Recent studies show that prompt tuning can better leverage the power of large language models than fine-tuning on downstream natural language understanding tasks. However, the existing prompt tuning methods have training instability issues, as the variance of scores under different random seeds is quite large. To address this critical problem, we first investigate and find that the loss landscape of vanilla prompt tuning is precipitous when it is visualized, where a slight change of input data can cause a big fluctuation in the loss landscape. This is an essential factor that leads to the instability of prompt tuning. Based on this observation, we introduce perturbation-based regularizers, which can smooth the loss landscape, into prompt tuning. We propose a new algorithm, called Prompt Tuning with Perturbation-based regularizer~(PTP), which can not only alleviate training instability dramatically but also boost the performance of prompt tuning. We design two kinds of perturbation-based regularizers, including random-noise-based and adversarial-based. In particular, our proposed perturbations are flexible on both text space and embedding space. Extensive experiments show the effectiveness of our proposed methods in stabilizing the training. Our new algorithms improve the state-of-the-art prompt tuning methods by 1.94\% and 2.34\% on SuperGLUE and FewGLUE benchmarks, respectively.

翻译：近期研究表明，在自然语言理解下游任务中，提示调优相比微调能更充分地发挥大型语言模型的能力。然而现有提示调优方法存在训练不稳定性问题，不同随机种子下的评分方差较大。针对这一关键问题，我们首先通过研究发现，标准提示调优的损失曲面在可视化时呈现陡峭特征，输入数据的微小变化即可引发损失曲面的显著波动。这是导致提示调优不稳定性的关键因素。基于此发现，我们将能平滑损失曲面的扰动正则化器引入提示调优。我们提出新算法——基于扰动正则化器的提示调优（PTP），该算法不仅能显著缓解训练不稳定性，还能提升提示调优性能。我们设计了随机噪声型和对抗型两类扰动正则化器，特别地，所提出的扰动在文本空间和嵌入空间中均具有灵活性。大量实验证明我们的方法在稳定训练方面的有效性。新算法在SuperGLUE和FewGLUE基准测试上分别将现有最优提示调优方法提升1.94%和2.34%。