Pretrained language models can be effectively stimulated by textual prompts or demonstrations, especially in low-data scenarios. Recent works have focused on automatically searching discrete or continuous prompts or optimized verbalizers, yet studies for the demonstration are still limited. Concretely, the demonstration examples are crucial for an excellent final performance of prompt-tuning. In this paper, we propose a novel pluggable, extensible, and efficient approach named contrastive demonstration tuning, which is free of demonstration sampling. Furthermore, the proposed approach can be: (i) Plugged into any previous prompt-tuning approaches; (ii) Extended to widespread classification tasks with a large number of categories. Experimental results on 16 datasets illustrate that our method integrated with previous approaches LM-BFF and P-tuning can yield better performance. Code is available in https://github.com/zjunlp/PromptKG/tree/main/research/Demo-Tuning.

翻译：预训练语言模型可以通过文本提示或示范样例得到有效激发，尤其在低数据场景下。已有研究主要聚焦于自动搜索离散或连续提示以及优化语言表达器，但对示范样例的研究仍较为有限。具体而言，示范样例对实现提示微调的最佳最终性能至关重要。本文提出一种名为对比示范调优的新型可插拔、可扩展且高效的方法，无需进行示范采样。此外，所提方法具有以下特性：（i）可嵌入任何现有的提示微调方法；（ii）可扩展至涵盖大量类别的广泛分类任务。在16个数据集上的实验结果表明，本方法集成现有LM-BFF和P-tuning方法后可获得更优性能。代码详见 https://github.com/zjunlp/PromptKG/tree/main/research/Demo-Tuning。