While textual information significantly enhances the performance of pre-trained language models (PLMs) in knowledge graph completion (KGC), the static and noisy nature of existing corpora collected from Wikipedia articles or synsets definitions often limits the potential of PLM-based KGC models. To surmount these challenges, we introduce the Contextualization Distillation strategy, a versatile plug-in-and-play approach compatible with both discriminative and generative KGC frameworks. Our method begins by instructing large language models (LLMs) to transform compact, structural triplets into context-rich segments. Subsequently, we introduce two tailored auxiliary tasks, reconstruction and contextualization, allowing smaller KGC models to assimilate insights from these enriched triplets. Comprehensive evaluations across diverse datasets and KGC techniques highlight the efficacy and adaptability of our approach, revealing consistent performance enhancements irrespective of underlying pipelines or architectures. Moreover, our analysis makes our method more explainable and provides insight into generating path selection, as well as the choosing of suitable distillation tasks. All the code and data in this work will be released at https://github.com/David-Li0406/Contextulization-Distillation

翻译：尽管文本信息显著提升了预训练语言模型（PLMs）在知识图谱补全（KGC）中的性能，但现有从维基百科文章或同义词集定义中收集的语料库的静态性和噪声特性，往往限制了基于PLM的KGC模型的潜力。为克服这些挑战，我们提出了语境化蒸馏策略，这是一种通用的即插即用方法，可兼容判别式和生成式KGC框架。我们的方法首先指导大语言模型（LLMs）将紧凑的结构化三元组转化为上下文丰富的片段。随后，我们引入了两个定制化的辅助任务——重构与语境化，使较小的KGC模型能够从这些增强的三元组中汲取见解。跨不同数据集和KGC技术的全面评估凸显了我们方法的有效性和适应性，揭示了无论底层流程或架构如何，性能均能实现持续提升。此外，我们的分析使该方法更具可解释性，并为生成路径选择及合适蒸馏任务的筛选提供了洞见。本工作的所有代码和数据将在 https://github.com/David-Li0406/Contextulization-Distillation 发布。