Quantifying variable importance is essential for answering high-stakes questions in fields like genetics, public policy, and medicine. Current methods generally calculate variable importance for a given model trained on a given dataset. However, for a given dataset, there may be many models that explain the target outcome equally well; without accounting for all possible explanations, different researchers may arrive at many conflicting yet equally valid conclusions given the same data. Additionally, even when accounting for all possible explanations for a given dataset, these insights may not generalize because not all good explanations are stable across reasonable data perturbations. We propose a new variable importance framework that quantifies the importance of a variable across the set of all good models and is stable across the data distribution. Our framework is extremely flexible and can be integrated with most existing model classes and global variable importance metrics. We demonstrate through experiments that our framework recovers variable importance rankings for complex simulation setups where other methods fail. Further, we show that our framework accurately estimates the true importance of a variable for the underlying data distribution. We provide theoretical guarantees on the consistency and finite sample error rates for our estimator. Finally, we demonstrate its utility with a real-world case study exploring which genes are important for predicting HIV load in persons with HIV, highlighting an important gene that has not previously been studied in connection with HIV. Code is available here.

翻译：摘要：量化变量重要性对于解答遗传学、公共政策及医学等领域中的高利害问题至关重要。当前方法通常针对给定数据集上训练的特定模型计算变量重要性。然而，对于同一数据集，可能存在多种同样能有效解释目标结果的模型；若未考虑所有可能解释，不同研究者可能基于相同数据得出相互矛盾但同样有效的结论。此外，即便考虑了给定数据集的所有可能解释，这些洞见也可能因并非所有良好解释在合理数据扰动下均保持稳定而无法泛化。我们提出一种新的变量重要性框架，该框架能跨所有良好模型集合量化变量的重要性，并在数据分布上保持稳定。我们的框架极为灵活，可与大多数现有模型类别及全局变量重要性度量方法集成。实验表明，在其他方法失效的复杂模拟场景中，我们的框架能恢复变量重要性排名。进一步，我们证明该框架能准确估计变量对底层数据分布的真实重要性。我们为该估计量的一致性及有限样本误差率提供了理论保证。最后，通过一项探索与HIV感染者病毒载量相关的关键基因的真实案例研究，我们展示了其实用价值，并突显了一个此前未在HIV关联研究中被报道的重要基因。代码已公开。