Artificial intelligence models and methods commonly lack causal interpretability. Despite the advancements in interpretable machine learning (IML) methods, they frequently assign importance to features which lack causal influence on the outcome variable. Selecting causally relevant features among those identified as relevant by these methods, or even before model training, would offer a solution. Feature selection methods utilizing information theoretical quantities have been successful in identifying statistically relevant features. However, the information theoretical quantities they are based on do not incorporate causality, rendering them unsuitable for such scenarios. To address this challenge, this article proposes information theoretical quantities that incorporate the causal structure of the system, which can be used to evaluate causal importance of features for some given outcome variable. Specifically, we introduce causal versions of entropy and mutual information, termed causal entropy and causal information gain, which are designed to assess how much control a feature provides over the outcome variable. These newly defined quantities capture changes in the entropy of a variable resulting from interventions on other variables. Fundamental results connecting these quantities to the existence of causal effects are derived. The use of causal information gain in feature selection is demonstrated, highlighting its superiority over standard mutual information in revealing which features provide control over a chosen outcome variable. Our investigation paves the way for the development of methods with improved interpretability in domains involving causation.

翻译：人工智能模型与方法普遍缺乏因果可解释性。尽管可解释机器学习（IML）方法取得了进展，但它们常常将缺乏对结果变量因果影响的特征赋予重要性。从这些方法识别出的相关特征中（甚至在模型训练之前）筛选出因果相关特征，是一种可行方案。基于信息论量的特征选择方法在识别统计相关特征方面已取得成功。然而，这些方法所依赖的信息论量未纳入因果关系，因此不适用于此类场景。为解决这一挑战，本文提出了融入系统因果结构的信息论量，可用于评估特征对特定结果变量的因果重要性。具体而言，我们引入了熵与互信息的因果版本——因果熵与因果信息增益，旨在量化特征对结果变量的控制程度。这些新定义量通过捕捉对变量实施干预后其熵的变化，建立了与因果效应存在性之间的基本联系。通过特征选择案例验证了因果信息增益相较于标准互信息在揭示哪些特征能控制目标结果变量方面的优越性。本研究为开发涉及因果领域时具有更强可解释性的方法奠定了基础。