Causal approaches to post-hoc explainability for black-box prediction models (e.g., deep neural networks trained on image pixel data) have become increasingly popular. However, existing approaches have two important shortcomings: (i) the "explanatory units" are micro-level inputs into the relevant prediction model, e.g., image pixels, rather than interpretable macro-level features that are more useful for understanding how to possibly change the algorithm's behavior, and (ii) existing approaches assume there exists no unmeasured confounding between features and target model predictions, which fails to hold when the explanatory units are macro-level variables. Our focus is on the important setting where the analyst has no access to the inner workings of the target prediction algorithm, rather only the ability to query the output of the model in response to a particular input. To provide causal explanations in such a setting, we propose to learn causal graphical representations that allow for arbitrary unmeasured confounding among features. We demonstrate the resulting graph can differentiate between interpretable features that causally influence model predictions versus those that are merely associated with model predictions due to confounding. Our approach is motivated by a counterfactual theory of causal explanation wherein good explanations point to factors that are "difference-makers" in an interventionist sense.

翻译：针对黑箱预测模型（例如基于图像像素数据训练的深度神经网络）的事后可解释性，因果方法日益流行。然而，现有方法存在两个重要缺陷：（i）"解释单元"是相关预测模型的微观层级输入（如图像像素），而非可解释的宏观层级特征——后者更有助于理解如何可能改变算法的行为；（ii）现有方法假设特征与目标模型预测之间不存在未测量的混杂因素，但当解释单元为宏观层级变量时这一假设不成立。我们的研究聚焦于一个关键场景：分析师无法访问目标预测算法的内部机制，仅能通过特定输入查询模型输出。在此场景下提供因果解释，我们提出学习允许特征间存在任意未测量混杂因素的因果图表示。实验表明，所得图能够区分两类可解释特征：因果性影响模型预测的特征，以及因混杂仅与模型预测存在关联的特征。本方法基于反事实因果解释理论——该理论认为好的解释应指向在干预主义意义上构成"差异制造者"的因素。