The ability to explain decisions made by machine learning models remains one of the most significant hurdles towards widespread adoption of AI in highly sensitive areas such as medicine, cybersecurity or autonomous driving. Great interest exists in understanding which features of the input data prompt model decision making. In this contribution, we propose a novel approach to identify relevant features of the input data, inspired by methods from the energy landscapes field, developed in the physical sciences. By identifying conserved weights within groups of minima of the loss landscapes, we can identify the drivers of model decision making. Analogues to this idea exist in the molecular sciences, where coordinate invariants or order parameters are employed to identify critical features of a molecule. However, no such approach exists for machine learning loss landscapes. We will demonstrate the applicability of energy landscape methods to machine learning models and give examples, both synthetic and from the real world, for how these methods can help to make models more interpretable.

翻译：机器学习模型决策解释能力的不足，仍是其在医学、网络安全、自动驾驶等高度敏感领域大规模应用面临的主要障碍之一。理解输入数据的哪些特征促使模型做出决策，已成为学界高度关注的课题。本文受物理科学中能量景观领域方法启发，提出了一种识别输入数据相关特征的新方法。通过识别损失景观极小值组中的守恒权重，我们可以定位模型决策的关键驱动因素。这一思路与分子科学中采用坐标不变量或序参量识别分子关键特征的方法具有相似性。然而，目前尚不存在适用于机器学习损失景观的此类方法。本文将论证能量景观方法在机器学习模型中的适用性，并通过人工合成数据与现实案例，说明如何运用这些方法提升模型可解释性。