Prompt-based classifiers are an attractive approach for zero-shot classification. However, the precise choice of the prompt template and label words can largely influence performance, with semantically equivalent settings often showing notable performance difference. This discrepancy can be partly attributed to word biases, where the classifier may be biased towards classes. To address this problem, it is possible to optimise classification thresholds on a labelled data set, however, this mitigates some of the advantages of prompt-based classifiers. This paper instead approaches this problem by examining the expected marginal probabilities of the classes. Here, probabilities are reweighted to have a uniform prior over classes, in an unsupervised fashion. Further, we draw a theoretical connection between the class priors and the language models' word prior, and offer the ability to set a threshold in a zero-resource fashion. We show that matching class priors correlates strongly with the oracle upper bound performance and demonstrate large consistent performance gains for prompt settings over a range of NLP tasks.

翻译：提示分类器是零样本分类任务的吸引力方法。然而，提示模板和标签词语的具体选择会显著影响性能，语义等价的设置往往表现出明显差异。这种差异可部分归因于词语偏差，即分类器可能偏向某些类别。为解决该问题，可以在标注数据集上优化分类阈值，但这削弱了提示分类器的部分优势。本文转而通过考察类别的期望边际概率来处理此问题。这里，概率被重新加权以实现无监督方式下的均匀先验分布。此外，我们从理论上建立了类别先验与语言模型词语先验之间的联系，并提出可在零资源条件下设定阈值。研究表明，匹配类别先验与预言上限性能高度相关，并在多项NLP任务的提示设置中展现了持续且显著的性能提升。