Variable importance plays a pivotal role in interpretable machine learning as it helps measure the impact of factors on the output of the prediction model. Model agnostic methods based on the generation of "null" features via permutation (or related approaches) can be applied. Such analysis is often utilized in pharmaceutical applications due to its ability to interpret black-box models, including tree-based ensembles. A major challenge and significant confounder in variable importance estimation however is the presence of between-feature correlation. Recently, several adjustments to marginal permutation utilizing feature knockoffs were proposed to address this issue, such as the variable importance measure known as conditional predictive impact (CPI). Assessment and evaluation of such approaches is the focus of our work. We first present a comprehensive simulation study investigating the impact of feature correlation on the assessment of variable importance. We then theoretically prove the limitation that highly correlated features pose for the CPI through the knockoff construction. While we expect that there is always no correlation between knockoff variables and its corresponding predictor variables, we prove that the correlation increases linearly beyond a certain correlation threshold between the predictor variables. Our findings emphasize the absence of free lunch when dealing with high feature correlation, as well as the necessity of understanding the utility and limitations behind methods in variable importance estimation.

翻译：变量重要性在可解释机器学习中发挥着关键作用，它有助于衡量各因素对预测模型输出的影响。基于置换（或相关方法）生成“空”特征模型不可知方法可应用于此。这类分析因能够解释包括树集成在内的黑箱模型，常被应用于制药领域。然而，变量重要性估计面临的主要挑战和显著干扰因素在于特征间的相关性。近期，针对边际置换的调整方案通过引入特征敲除（knockoffs）技术来应对该问题，例如条件预测影响（CPI）这一变量重要性度量指标。本研究重点评估此类方法。我们首先开展全面的模拟研究，探讨特征相关性对变量重要性评估的影响；随后从理论上证明，通过敲除构造（knockoff construction）机制，高度相关特征对CPI存在局限性。尽管我们预期敲除变量与其对应预测变量之间始终无相关性，但证明当预测变量间的相关性超过某个阈值后，相关性会呈线性增长。研究结果强调，在处理高特征相关性时不存在免费午餐，同时揭示了理解变量重要性估计方法效用与局限性的必要性。