Pre-trained language models (PLMs) have achieved strong performance in keyphrase extraction (KPE), largely due to their ability to generate rich contextualized representations. However, long-document KPE remains challenging because salient keyphrase evidence may be scattered across distant document sections that cannot be jointly captured within the limited context window of most PLMs. Although long-context large language models (LLMs) can process broader textual contexts, their computational cost limits their practicality for efficient and high-throughput KPE. To overcome this limitation, we propose an attention expansion mechanism that augments PLM token representations with information from surrounding out-of-context chunks using pre-trained word embeddings. The proposed mechanism expands the effective contextual scope of PLM-based KPE models without requiring full-document attention or expensive LLM-based inference. We evaluate our approach across five PLM backbones, including general-purpose, scientific, task-specific, and long-context encoders, using two training regimes and five benchmark corpora from scientific and news domains. Experimental results demonstrate that attention expansion consistently enhances KPE performance across all evaluation settings, outperforming state-of-the-art models and yielding notable improvements in F1 score. The improvements extend to domain-specific, task-specialized, and native long-context models, showing that the proposed mechanism provides complementary information rather than merely compensating for limited input length. These results establish attention expansion as an efficient and effective strategy for long-document KPE.

翻译：预训练语言模型（PLMs）凭借其生成丰富上下文表征的能力，在关键词抽取任务中取得了优异表现。然而，长文档关键词抽取仍面临挑战，因为显著的关键词证据可能分散在文档的远距离段落中，而大多数PLMs的有限上下文窗口无法同时捕获这些信息。尽管长上下文大语言模型（LLMs）能处理更广阔的文本语境，但其计算成本限制了其在高效高吞吐量关键词抽取中的实用性。为解决这一局限，我们提出注意力扩展机制，通过预训练词嵌入将周围非上下文片段的信息融入PLM的token表征。该机制在不依赖全文档注意力或昂贵的LLM推理的情况下，扩展了基于PLM的关键词抽取模型的有效上下文范围。我们在五种PLM骨干模型（包括通用型、科学型、任务专用型及长上下文编码器）上，采用两种训练范式及来自科学和新闻领域的五个基准语料库进行评估。实验结果表明，注意力扩展在所有评估设置中均能持续提升关键词抽取性能，不仅超越现有最优模型，且在F1分数上取得显著改进。这些改进同样适用于领域专用、任务特化及原生长上下文模型，表明所提机制并非仅通过弥补输入长度限制发挥作用，而是提供了互补性信息。上述结果确立了注意力扩展作为长文档关键词抽取的高效且有效策略。