Commonsense reasoning in natural language is a desired ability of artificial intelligent systems. For solving complex commonsense reasoning tasks, a typical solution is to enhance pre-trained language models~(PTMs) with a knowledge-aware graph neural network~(GNN) encoder that models a commonsense knowledge graph~(CSKG). Despite the effectiveness, these approaches are built on heavy architectures, and can't clearly explain how external knowledge resources improve the reasoning capacity of PTMs. Considering this issue, we conduct a deep empirical analysis, and find that it is indeed relation features from CSKGs (but not node features) that mainly contribute to the performance improvement of PTMs. Based on this finding, we design a simple MLP-based knowledge encoder that utilizes statistical relation paths as features. Extensive experiments conducted on five benchmarks demonstrate the effectiveness of our approach, which also largely reduces the parameters for encoding CSKGs. Our codes and data are publicly available at https://github.com/RUCAIBox/SAFE.

翻译：自然语言的常识推理是人工智能系统的一种理想能力。为了解决复杂的常识推理任务,典型的解决方案是用一个知识-觉悟图形神经网络~(GNN)编码器来强化预先训练的语言模型~(PTMs),该模型模型将模拟一个常识知识图~(CSKG ) 。尽管具有效力,但这些方法建立在重体型结构上,无法清楚地解释外部知识资源如何提高PTM的推理能力。考虑到这一问题,我们进行了深入的经验分析,发现这确实是CSKGs(但非节点特征)的关联特征,主要有助于改进PTMs的性能。基于这一发现,我们设计了一个简单的基于MLP的知识编码器,利用统计关系路径作为特征。在五个基准上进行的广泛实验显示了我们的方法的有效性,这也大大降低了CSKGs的编码参数。我们的代码和数据在https://github.com/RUCABox/SAFE上公开提供。