Commonsense knowledge is crucial to many natural language processing tasks. Existing works usually incorporate graph knowledge with conventional graph neural networks (GNNs), leading to the text and graph knowledge encoding processes being separated in a serial pipeline. We argue that these separate representation learning stages may be suboptimal for neural networks to learn the overall context contained in both types of input knowledge. In this paper, we propose a novel context-aware graph-attention model (Context-aware GAT), which can effectively incorporate global features of relevant knowledge graphs based on a context-enhanced knowledge aggregation process. Specifically, our framework leverages a novel representation learning approach to process heterogeneous features - combining flattened graph knowledge with text. To the best of our knowledge, this is the first attempt at hierarchically applying graph knowledge aggregation on a connected subgraph in addition to contextual information to support commonsense dialogue generation. This framework shows superior performance compared to conventional GNN-based language frameworks. Both automatic and human evaluation demonstrates that our proposed model has significant performance uplifts over state-of-the-art baselines.

翻译：常识知识对于许多自然语言处理任务至关重要。现有方法通常利用传统的图神经网络（GNN）处理图知识，导致文本和图知识的编码过程在串行流水线中相互分离。我们认为，这些分离的表征学习阶段可能不利于神经网络充分学习两类输入知识中蕴含的整体上下文信息。本文提出了一种新颖的上下文感知图注意力模型（Context-aware GAT），该模型基于上下文增强的知识聚合过程，能够有效融合相关知识图谱的全局特征。具体而言，我们的框架采用了一种新颖的表征学习方法处理异构特征——将展平的图知识与文本相结合。据我们所知，这是首次尝试在连接子图上分层应用图知识聚合与上下文信息，以支持常识对话生成。该框架相较传统基于GNN的语言框架展现出更优性能。自动评估与人工评估均证明，我们提出的模型相较于目前最优基线方法取得了显著的性能提升。