Commonsense knowledge is crucial to many natural language processing tasks. Existing works usually incorporate graph knowledge with conventional graph neural networks (GNNs), leading to the text and graph knowledge encoding processes being separated in a serial pipeline. We argue that these separate representation learning stages may be suboptimal for neural networks to learn the overall context contained in both types of input knowledge. In this paper, we propose a novel context-aware graph-attention model (Context-aware GAT), which can effectively incorporate global features of relevant knowledge graphs based on a context-enhanced knowledge aggregation process. Specifically, our framework leverages a novel representation learning approach to process heterogeneous features - combining flattened graph knowledge with text. To the best of our knowledge, this is the first attempt at hierarchically applying graph knowledge aggregation on a connected subgraph in addition to contextual information to support commonsense dialogue generation. This framework shows superior performance compared to conventional GNN-based language frameworks. Both automatic and human evaluation demonstrates that our proposed model has significant performance uplifts over state-of-the-art baselines.

翻译：常识知识对许多自然语言处理任务至关重要。现有工作通常使用传统图神经网络（GNN）整合图知识，导致文本和图知识的编码过程在串行流水线中分离。我们主张这种分离的表示学习阶段可能不利于神经网络学习两种输入知识中包含的整体上下文。本文提出一种新颖的上下文感知图注意力模型（Context-aware GAT），通过上下文增强的知识聚合过程有效整合相关知识图的全局特征。具体而言，我们的框架采用新颖的表示学习方法处理异构特征——将展平图知识与文本结合。据我们所知，这是首次尝试在上下文信息之外，对连接子图分层应用图知识聚合以支持常识对话生成。该框架相比传统基于GNN的语言框架表现出更优性能。自动评测和人工评测均表明，我们提出的模型相比最先进的基线方法有显著性能提升。