Prompt-based learning has shown its effectiveness in few-shot text classification. One important factor in its success is a verbalizer, which translates output from a language model into a predicted class. Notably, the simplest and widely acknowledged verbalizer employs manual labels to represent the classes. However, manual selection does not guarantee the optimality of the selected words when conditioned on the chosen language model. Therefore, we propose Label-Aware Automatic Verbalizer (LAAV), effectively augmenting the manual labels to achieve better few-shot classification results. Specifically, we use the manual labels along with the conjunction "and" to induce the model to generate more effective words for the verbalizer. The experimental results on five datasets across five languages demonstrate that LAAV significantly outperforms existing verbalizers. Furthermore, our analysis reveals that LAAV suggests more relevant words compared to similar approaches, especially in mid-to-low resource languages.
翻译:基于提示的学习在小样本文本分类中已展现出其有效性。其成功的关键因素之一是动词化器,该组件负责将语言模型的输出转化为预测类别。值得注意的是,最简单且被广泛认可的动词化器采用人工标签来表征类别。然而,人工选择无法保证所选词语在给定语言模型条件下的最优性。为此,我们提出标签感知自动动词化器(LAAV),通过有效增强人工标签以实现更优的小样本分类效果。具体而言,我们利用人工标签与连词"and"引导模型生成更高效的动词化器词汇。在五种语言数据集上的实验结果表明,LAAV显著优于现有动词化器。进一步分析显示,与同类方法相比,LAAV能够生成更相关的词汇,尤其在低资源语言场景中表现突出。