Recent prompt optimisation approaches use the generative nature of language models to produce prompts -- even rivaling the performance of human-curated prompts. In this paper, we demonstrate that randomly sampling tokens from the model vocabulary as ``separators'' can be as effective as language models for prompt-style text classification. Our experiments show that random separators are competitive baselines, having less than a 1% difference compared to previous self-optimisation methods and showing a 12% average relative improvement over strong human baselines across nine text classification tasks and eight language models. We further analyse this phenomenon in detail using three different random generation strategies, establishing that the language space is rich with potentially good separators, with a greater than 40% average chance that a randomly drawn separator performs better than human-curated separators. These observations challenge the common assumption that an effective prompt should be human readable or task relevant and establish a strong baseline for prompt optimisation research.

翻译：最近的提示优化方法利用语言模型的生成特性来产生提示——甚至能与人工设计的提示性能相媲美。在本文中，我们证明从模型词汇表中随机采样词元作为“分隔符”在提示式文本分类中可以与语言模型同样有效。我们的实验表明，随机分隔符是竞争性基线，与之前的自优化方法相比差异小于1%，在九个文本分类任务和八个语言模型上，相比强人工基线平均相对改进12%。我们进一步使用三种不同的随机生成策略详细分析这一现象，证实语言空间中蕴含丰富的潜在优质分隔符，随机抽取的分隔符平均有超过40%的概率优于人工设计的分隔符。这些观察结果挑战了“有效提示应具有人类可读性或任务相关性”的普遍假设，并为提示优化研究建立了强基线。