Text generation from Abstract Meaning Representation (AMR) has substantially benefited from the popularized Pretrained Language Models (PLMs). Myriad approaches have linearized the input graph as a sequence of tokens to fit the PLM tokenization requirements. Nevertheless, this transformation jeopardizes the structural integrity of the graph and is therefore detrimental to its resulting representation. To overcome this issue, Ribeiro et al. have recently proposed StructAdapt, a structure-aware adapter which injects the input graph connectivity within PLMs using Graph Neural Networks (GNNs). In this paper, we investigate the influence of Relative Position Embeddings (RPE) on AMR-to-Text, and, in parallel, we examine the robustness of StructAdapt. Through ablation studies, graph attack and link prediction, we reveal that RPE might be partially encoding input graphs. We suggest further research regarding the role of RPE will provide valuable insights for Graph-to-Text generation.

翻译：从抽象语义表示（AMR）生成文本已从流行的预训练语言模型（PLMs）中显著受益。众多方法将输入图线性化为令牌序列，以符合PLM的令牌化要求。然而，这种转换破坏了图的结构完整性，从而对其生成的表示产生不利影响。为解决这一问题，Ribeiro等人近期提出了StructAdapt，一种通过图神经网络（GNNs）将输入图连通性注入PLMs的结构感知适配器。本文研究了相对位置嵌入（RPE）对AMR到文本生成的影响，并同时考察了StructAdapt的鲁棒性。通过消融实验、图攻击和链接预测，我们发现RPE可能部分编码了输入图。我们建议进一步研究RPE的作用，这将为图到文本生成提供有价值的见解。