Keyphrases which are useful in several NLP and IR applications are either extracted from text or predicted by generative models. Contrarily to keyphrase extraction approaches, keyphrase generation models can predict keyphrases that do not appear in a document's text called `absent keyphrases`. This ability means that keyphrase generation models can associate a document to a notion that is not explicitly mentioned in its text. Intuitively, this suggests that for two documents treating the same subjects, a keyphrase generation model is more likely to be homogeneous in their indexing i.e. predict the same keyphrase for both documents, regardless of those keyphrases appearing in their respective text or not; something a keyphrase extraction model would fail to do. Yet, homogeneity of keyphrase prediction models is not covered by current benchmarks. In this work, we introduce a method to evaluate the homogeneity of keyphrase prediction models and study if absent keyphrase generation capabilities actually help the model to be more homogeneous. To our surprise, we show that keyphrase extraction methods are competitive with generative models, and that the ability to generate absent keyphrases can actually have a negative impact on homogeneity. Our data, code and prompts are available on huggingface and github.

翻译：关键词在多种自然语言处理和信息检索应用中具有重要作用，可通过文本抽取或生成模型预测获得。与关键词抽取方法不同，关键词生成模型能够预测文档文本中未出现的所谓“缺失关键词”。这种能力意味着关键词生成模型能够将文档与其文本未明确提及的概念相关联。直观而言，这表明对于处理相同主题的两份文档，关键词生成模型在索引时更可能保持同质性——即对两份文档预测相同的关键词，无论这些关键词是否出现在各自文本中；而关键词抽取模型则无法做到这一点。然而，当前基准测试尚未涵盖关键词预测模型的同质性评估。本研究提出了一种评估关键词预测模型同质性的方法，并探究缺失关键词生成能力是否实际有助于提升模型的同质性。令人意外的是，我们发现关键词抽取方法与生成模型相比具有竞争力，且生成缺失关键词的能力实际上可能对同质性产生负面影响。我们的数据、代码及提示已发布于huggingface和github平台。