The black-box nature of complex Neural Network (NN)-based models has hindered their widespread adoption in security domains due to the lack of logical explanations and actionable follow-ups for their predictions. To enhance the transparency and accountability of Graph Neural Network (GNN) security models used in system provenance analysis, we propose PROVEXPLAINER, a framework for projecting abstract GNN decision boundaries onto interpretable feature spaces. We first replicate the decision-making process of GNNbased security models using simpler and explainable models such as Decision Trees (DTs). To maximize the accuracy and fidelity of the surrogate models, we propose novel graph structural features founded on classical graph theory and enhanced by extensive data study with security domain knowledge. Our graph structural features are closely tied to problem-space actions in the system provenance domain, which allows the detection results to be explained in descriptive, human language. PROVEXPLAINER allowed simple DT models to achieve 95% fidelity to the GNN on program classification tasks with general graph structural features, and 99% fidelity on malware detection tasks with a task-specific feature package tailored for direct interpretation. The explanations for malware classification are demonstrated with case studies of five real-world malware samples across three malware families.

翻译：基于复杂神经网络（NN）的模型因其黑箱特性而缺乏逻辑解释和可操作的预测后续处理，严重阻碍了其在安全领域的广泛应用。为增强图神经网络（GNN）安全模型在系统溯源分析中的透明度和问责性，我们提出PROVEXPLAINER框架，该框架能够将抽象的GNN决策边界映射到可解释的特征空间。我们首先利用决策树（DT）等更简单且可解释的模型来复现基于GNN的安全模型的决策过程。为最大化代理模型的准确性和保真度，我们提出了基于经典图论且经过安全领域知识数据深度增强的新型图结构特征。这些图结构特征与系统溯源领域的问题空间操作紧密关联，从而能够以描述性的人类语言解释检测结果。实验表明，采用通用图结构特征时，PROVEXPLAINER使简单DT模型在程序分类任务上达到与GNN 95%的保真度；在恶意软件检测任务中，通过定制面向直接解释的任务特定特征包，保真度提升至99%。我们通过涵盖三个恶意软件家族的五个真实样本案例研究，展示了恶意软件分类的解释效果。