Prompt tuning is a parameter-efficient method, which freezes all PLM parameters and only prepends some additional tunable tokens called soft prompts to the input text. However, soft prompts heavily rely on a better initialization and may easily result in overfitting under few-shot settings, which causes prompt-tuning performing much worse than fine-tuning. To address the above issues, this paper proposes a novel Self-sUpervised Meta-prompt learning framework with MEtagradient Regularization for few shot generalization (SUMMER). We leverage self-supervised meta-learning to better initialize soft prompts and curriculum-based task augmentation is further proposed to enrich the meta-task distribution. Besides, a novel meta-gradient regularization method is integrated into the meta-prompt learning framework, which meta-learns to transform the raw gradient during few-shot learning into a domain-generalizable direction, thus alleviating the problem of overfitting. Extensive experiments show that SUMMER achieves better performance for different few-shot downstream tasks, and also exhibits a stronger domain generalization ability.

翻译：提示调优是一种参数高效方法，它冻结所有预训练语言模型参数，仅在输入文本前添加少量可调优的令牌（称为软提示）。然而，软提示严重依赖于更优的初始化，且在少样本设置下容易过拟合，导致提示调优性能远低于微调。针对上述问题，本文提出一种新颖的基于元梯度正则化的自监督元提示学习框架（SUMMER），用于少样本泛化。我们利用自监督元学习来更好地初始化软提示，并进一步提出基于课程的任务增强以丰富元任务分布。此外，一种新颖的元梯度正则化方法被集成到元提示学习框架中，该方法通过元学习将少样本学习过程中的原始梯度转换为领域可泛化的方向，从而缓解过拟合问题。大量实验表明，SUMMER在多种少样本下游任务中取得更优性能，同时展现出更强的领域泛化能力。