Large language models (LMs) such as GPT-4 are very powerful and can process different kinds of natural language processing (NLP) tasks. However, it can be difficult to interpret the results due to the multi-layer nonlinear model structure and millions of parameters. Lack of understanding of how the model works can make the model unreliable and dangerous for everyday users in real-world scenarios. Most recent works exploit the weights of attention to provide explanations for model predictions. However, pure attention-based explanation is unable to support the growing complexity of the models, and cannot reason about their decision-making processes. Thus, we propose LMExplainer, a knowledge-enhanced interpretation module for language models that can provide human-understandable explanations. We use a knowledge graph (KG) and a graph attention neural network to extract the key decision signals of the LM. We further explore whether interpretation can also help AI understand the task better. Our experimental results show that LMExplainer outperforms existing LM+KG methods on CommonsenseQA and OpenBookQA. We also compare the explanation results with generated explanation methods and human-annotated results. The comparison shows our method can provide more comprehensive and clearer explanations. LMExplainer demonstrates the potential to enhance model performance and furnish explanations for the reasoning processes of models in natural language.

翻译：大型语言模型（如GPT-4）功能强大，能够处理各类自然语言处理任务。然而，由于多层非线性模型结构和数百万参数的存在，其结果的解释性较差。缺乏对模型工作机制的理解，会使模型在现实场景中对于日常用户而言变得不可靠且存在风险。近期工作多利用注意力权重为模型预测提供解释，但纯注意力机制的解释无法支撑日益复杂的模型，也无法推理其决策过程。为此，我们提出LMExplainer——一种知识增强的语言模型解释模块，可提供人类可理解的解释。我们采用知识图谱与图注意力神经网络提取语言模型的关键决策信号，并进一步探究解释是否有助于提升人工智能对任务的理解。实验结果表明，LMExplainer在CommonsenseQA和OpenBookQA数据集上优于现有"语言模型+知识图谱"方法。我们还将解释结果与生成式解释方法及人工标注结果进行了对比，显示本方法能提供更全面、清晰的解释。LMExplainer展现了增强模型性能并为自然语言推理过程提供解释的潜力。