Prompt recently have become an effective linguistic tool on utilizing the pre-trained language models. However, in few-shot scenarios, subtle changes of prompt's design always make the result widely different, and the prompt design is also easy to overfit the current limited samples. To alleviate this, we explore how to utilize suitable contrastive samples and multiple contrastive learning methods to realize a more robust prompt's representation. Therefore, the contrastive prompt model ConsPrompt combining with prompt encoding network, contrastive sampling modules, and contrastive scoring modules are introduced to realize differential contrastive learning. Our results exhibit the state-of-the-art performance in different few-shot settings, and the ablation experiments also certificate the effectiveness in utilizing multi-degree contrastive learning in prompt-based fine-tuning process.

翻译：提示学习近年来已成为利用预训练语言模型的有效语言工具。然而，在小样本场景下，提示设计的细微变化常导致结果差异显著，且提示设计易过度拟合当前有限的样本。为缓解此问题，我们探索如何利用合适的对比样本及多种对比学习方法，以实现更鲁棒的提示表示。为此，我们引入了结合提示编码网络、对比采样模块与对比评分模块的对比提示模型ConsPrompt，以实现差异化对比学习。实验结果表明，在不同小样本设置下，我们的模型均取得了最先进的性能，消融实验也验证了基于多程度对比学习在提示微调过程中的有效性。