Recent advances for few-shot text classification aim to wrap textual inputs with task-specific prompts to cloze questions. By processing them with a masked language model to predict the masked tokens and using a verbalizer that constructs the mapping between predicted words and target labels. This approach of using pre-trained language models is called prompt-based tuning, which could remarkably outperform conventional fine-tuning approach in the low-data scenario. As the core of prompt-based tuning, the verbalizer is usually handcrafted with human efforts or suboptimally searched by gradient descent. In this paper, we focus on automatically constructing the optimal verbalizer and propose a novel evolutionary verbalizer search (EVS) algorithm, to improve prompt-based tuning with the high-performance verbalizer. Specifically, inspired by evolutionary algorithm (EA), we utilize it to automatically evolve various verbalizers during the evolutionary procedure and select the best one after several iterations. Extensive few-shot experiments on five text classification datasets show the effectiveness of our method.

翻译：近期在少样本文本分类领域的进展主要围绕将文本输入与任务特定提示相结合，形成完形填空问题。通过掩码语言模型处理以预测被掩码的标记，并利用语言化器构建预测词与目标标签之间的映射。这种使用预训练语言模型的方法称为基于提示的微调，在低数据场景下显著优于传统微调方法。作为基于提示微调的核心，语言化器通常需要人工精心设计，或通过梯度下降进行次优搜索。本文聚焦于自动构建最优语言化器，提出一种新颖的进化性语言化器搜索算法，旨在通过高性能语言化器提升基于提示的微调性能。具体而言，受进化算法启发，我们利用其在进化过程中自动演化多种语言化器，并在多次迭代后选择最优结果。在五个文本分类数据集上的大量少样本实验验证了该方法的有效性。