With the increased deployment of machine learning models in various real-world applications, researchers and practitioners alike have emphasized the need for explanations of model behaviour. To this end, two broad strategies have been outlined in prior literature to explain models. Post hoc explanation methods explain the behaviour of complex black-box models by highlighting features that are critical to model predictions; however, prior work has shown that these explanations may not be faithful, and even more concerning is our inability to verify them. Specifically, it is nontrivial to evaluate if a given attribution is correct with respect to the underlying model. Inherently interpretable models, on the other hand, circumvent these issues by explicitly encoding explanations into model architecture, meaning their explanations are naturally faithful and verifiable, but they often exhibit poor predictive performance due to their limited expressive power. In this work, we aim to bridge the gap between the aforementioned strategies by proposing Verifiability Tuning (VerT), a method that transforms black-box models into models that naturally yield faithful and verifiable feature attributions. We begin by introducing a formal theoretical framework to understand verifiability and show that attributions produced by standard models cannot be verified. We then leverage this framework to propose a method to build verifiable models and feature attributions out of fully trained black-box models. Finally, we perform extensive experiments on semi-synthetic and real-world datasets, and show that VerT produces models that (1) yield explanations that are correct and verifiable and (2) are faithful to the original black-box models they are meant to explain.

翻译：随着机器学习模型在各行各业实际应用中的广泛部署，研究者和从业者均强调对模型行为进行解释的必要性。为此，既有文献概述了两大类解释策略：事后解释方法通过突出对模型预测至关重要的特征来解释复杂黑箱模型的行为；然而，先前研究表明这些解释可能不够忠实，更令人担忧的是我们无法对其进行验证——具体而言，很难评估某个归因结果相对于底层模型是否正确。相比之下，内在可解释模型通过将解释直接编码进模型架构来规避这些问题，这意味着其解释天然具有忠实性与可验证性，但由于表达能力受限，此类模型往往预测性能较差。本研究旨在通过提出可验证性调优（VerT）方法弥合上述两类策略的差距，该方法可将黑箱模型转化为能够自然产生忠实且可验证特征归因的模型。我们首先引入形式化的理论框架来理解可验证性，并证明标准模型产生的归因结果无法被验证。继而利用该框架提出方法，从完全训练好的黑箱模型出发构建可验证模型及特征归因。最后，我们在半合成与真实数据集上开展大量实验，结果表明VerT生成的模型（1）能产生正确且可验证的解释，（2）对其所要解释的原始黑箱模型保持忠实性。