Recent studies have demonstrated that natural-language prompts can help to leverage the knowledge learned by pre-trained language models for the binary sentence-level sentiment classification task. Specifically, these methods utilize few-shot learning settings to fine-tune the sentiment classification model using manual or automatically generated prompts. However, the performance of these methods is sensitive to the perturbations of the utilized prompts. Furthermore, these methods depend on a few labeled instances for automatic prompt generation and prompt ranking. This study aims to find high-quality prompts for the given task in a zero-shot setting. Given a base prompt, our proposed approach automatically generates multiple prompts similar to the base prompt employing positional, reasoning, and paraphrasing techniques and then ranks the prompts using a novel metric. We empirically demonstrate that the top-ranked prompts are high-quality and significantly outperform the base prompt and the prompts generated using few-shot learning for the binary sentence-level sentiment classification task.
翻译:近期研究表明,自然语言提示有助于利用预训练语言模型所学知识,完成二元句子级情感分类任务。具体而言,这些方法采用少样本学习设置,通过人工或自动生成的提示对情感分类模型进行微调。然而,这些方法的性能对所采用提示的扰动较为敏感。此外,这些方法依赖少量标注实例以自动生成提示并进行提示排序。本研究旨在零样本设置下为给定任务寻找高质量提示。基于基础提示,我们提出的方法通过位置调整、推理增强及转述改写等技术自动生成多个相似提示,并采用新型评估指标进行排序。实验证明,排名前列的提示具有高质量,且其性能显著优于基础提示及基于少样本学习生成的提示,在二元句子级情感分类任务中表现突出。