Recent improvements in KG-to-text generation are due to additional auxiliary pre-training tasks designed to give the fine-tune task a boost in performance. These tasks require extensive computational resources while only suggesting marginal improvements. Here, we demonstrate that by fusing graph-aware elements into existing pre-trained language models, we are able to outperform state-of-the-art models and close the gap imposed by additional pre-training tasks. We do so by proposing a mask structure to capture neighborhood information and a novel type encoder that adds a bias to the graph-attention weights depending on the connection type. Experiments on two KG-to-text benchmark datasets show our models are competitive while involving fewer parameters and no additional pre-training tasks. By formulating the problem as a framework, we can interchange the various proposed components and begin interpreting KG-to-text generative models based on the topological and type information found in a graph.

翻译：近年来，知识图谱到文本生成的改进得益于额外的辅助预训练任务，这些任务旨在提升微调阶段的性能。然而，此类任务需要大量计算资源，且仅带来边际性改善。本文证明，通过将图感知元素融入现有预训练语言模型，我们能够超越现有最优模型，并弥合因额外预训练任务造成的性能差距。为此，我们提出一种用于捕获邻域信息的掩码结构，以及一种根据连接类型为图注意力权重添加偏置的新型类型编码器。在两个知识图谱到文本基准数据集上的实验表明，我们的模型在参数更少且无需额外预训练任务的情况下仍具有竞争力。通过将问题形式化为一个框架，我们可以灵活替换各组件，并基于图中拓扑信息与类型信息对知识图谱到文本生成模型进行解释。