Edges in many real-world social/information networks are associated with rich text information (e.g., user-user communications or user-product reviews). However, mainstream network representation learning models focus on propagating and aggregating node attributes, lacking specific designs to utilize text semantics on edges. While there exist edge-aware graph neural networks, they directly initialize edge attributes as a feature vector, which cannot fully capture the contextualized text semantics of edges. In this paper, we propose Edgeformers, a framework built upon graph-enhanced Transformers, to perform edge and node representation learning by modeling texts on edges in a contextualized way. Specifically, in edge representation learning, we inject network information into each Transformer layer when encoding edge texts; in node representation learning, we aggregate edge representations through an attention mechanism within each node's ego-graph. On five public datasets from three different domains, Edgeformers consistently outperform state-of-the-art baselines in edge classification and link prediction, demonstrating the efficacy in learning edge and node representations, respectively.

翻译：在许多现实世界的社会/信息网络中，边通常关联着丰富的文本信息（例如用户间的通信或用户对产品的评价）。然而，主流的网络表示学习模型侧重于传播和聚合节点属性，缺乏利用边文本语义的专门设计。尽管存在边感知的图神经网络，但它们将边属性直接初始化为特征向量，无法充分捕捉边的上下文化文本语义。本文提出Edgeformers框架，该框架基于图增强的Transformer，通过以上下文方式对边文本进行建模，实现边和节点的表示学习。具体而言，在边表示学习中，我们在编码边文本时将网络信息注入每个Transformer层；在节点表示学习中，通过节点自我图内的注意力机制聚合边表示。在来自三个不同领域的五个公共数据集上，Edgeformers在边分类和链接预测任务中均持续优于最先进的基线模型，分别证明了其在边和节点表示学习中的有效性。