Current methods for prompt learning in zeroshot scenarios widely rely on a development set with sufficient human-annotated data to select the best-performing prompt template a posteriori. This is not ideal because in a realworld zero-shot scenario of practical relevance, no labelled data is available. Thus, we propose a simple yet effective method for screening reasonable prompt templates in zero-shot text classification: Perplexity Selection (Perplection). We hypothesize that language discrepancy can be used to measure the efficacy of prompt templates, and thereby develop a substantiated perplexity-based scheme allowing for forecasting the performance of prompt templates in advance. Experiments show that our method leads to improved prediction performance in a realistic zero-shot setting, eliminating the need for any labelled examples.

翻译：当前零样本场景下的提示学习方法广泛依赖于一个包含充足人工标注数据的开发集，用于事后选择性能最优的提示模板。这在现实相关且具有实际应用价值的零样本场景中并不理想，因为此时没有任何标注数据可用。为此，我们提出了一种简单而有效的方法，用于在零样本文本分类中筛选合理的提示模板：困惑度选择法（Perplection）。我们假设语言差异可用于衡量提示模板的有效性，并由此构建了一套基于困惑度的可验证方案，能够预先预测提示模板的性能。实验表明，在真实的零样本设置下，我们的方法提升了预测性能，且无需任何标注样本。