Artificial intelligence models and methods commonly lack causal interpretability. Despite the advancements in interpretable machine learning (IML) methods, they frequently assign importance to features which lack causal influence on the outcome variable. Selecting causally relevant features among those identified as relevant by these methods, or even before model training, would offer a solution. Feature selection methods utilizing information theoretical quantities have been successful in identifying statistically relevant features. However, the information theoretical quantities they are based on do not incorporate causality, rendering them unsuitable for such scenarios. To address this challenge, this article proposes information theoretical quantities that incorporate the causal structure of the system, which can be used to evaluate causal importance of features for some given outcome variable. Specifically, we introduce causal versions of entropy and mutual information, termed causal entropy and causal information gain, which are designed to assess how much control a feature provides over the outcome variable. These newly defined quantities capture changes in the entropy of a variable resulting from interventions on other variables. Fundamental results connecting these quantities to the existence of causal effects are derived. The use of causal information gain in feature selection is demonstrated, highlighting its superiority over standard mutual information in revealing which features provide control over a chosen outcome variable. Our investigation paves the way for the development of methods with improved interpretability in domains involving causation.

翻译：人工智能模型与方法通常缺乏因果可解释性。尽管可解释机器学习方法已取得进展，但这类方法常将权重赋予对结果变量缺乏因果影响的特征。从这些方法识别的相关特征中（甚至是在模型训练前）筛选出具有因果相关性的特征，将为此提供解决方案。基于信息论量的特征选择方法已成功识别出统计相关特征，但这些方法所依赖的信息论量并未融合因果性，因而难以适用于此类场景。为攻克这一难题，本文提出融合系统因果结构的信息论量，可用于评估特定结果变量下特征的因果重要性。具体而言，我们引入了因果熵与因果互信息（分别称为因果熵与因果信息增益），旨在衡量特征对结果变量的控制程度。这些新定义量可捕捉因对其他变量实施干预而导致的变量熵值变化，并推导出连接这些量与因果效应存在性的基本定理。本文展示了因果信息增益在特征选择中的应用，揭示其在揭示哪些特征能控制选定结果变量方面优于标准互信息。本研究为开发因果领域更具可解释性的方法奠定了基础。