Black box models only provide results for deep learning tasks, and lack informative details about how these results were obtained. Knowing how input variables are related to outputs, in addition to why they are related, can be critical to translating predictions into laboratory experiments, or defending a model prediction under scrutiny. In this paper, we propose a general theory that defines a variance tolerance factor (VTF) inspired by influence function, to interpret features in the context of black box neural networks by ranking the importance of features, and construct a novel architecture consisting of a base model and feature model to explore the feature importance in a Rashomon set that contains all well-performing neural networks. Two feature importance ranking methods in the Rashomon set and a feature selection method based on the VTF are created and explored. A thorough evaluation on synthetic and benchmark datasets is provided, and the method is applied to two real world examples predicting the formation of noncrystalline gold nanoparticles and the chemical toxicity 1793 aromatic compounds exposed to a protozoan ciliate for 40 hours.

翻译：黑箱模型仅提供深度学习任务的结果，缺乏关于这些结果如何获得的详细信息。了解输入变量与输出之间的关系，以及为何存在这种关系，对于将预测转化为实验室实验或在审查中捍卫模型预测至关重要。本文提出了一种通用理论，受影响函数启发定义了方差容限因子（VTF），通过特征重要性排序来解释黑箱神经网络中的特征，并构建了一种由基础模型和特征模型组成的新型架构，以探索包含所有高性能神经网络的Rashomon集合中的特征重要性。我们创建并研究了Rashomon集合中的两种特征重要性排序方法以及基于VTF的特征选择方法。在合成数据集和基准数据集上进行了全面评估，并将该方法应用于两个真实世界案例：预测非晶金纳米颗粒的形成以及1793种芳香族化合物对原生动物纤毛虫暴露40小时的化学毒性。