Feature importance inference is critical for the interpretability and reliability of machine learning models. There has been increasing interest in developing model-agnostic approaches to interpret any predictive model, often in the form of feature occlusion or leave-one-covariate-out (LOCO) inference. Existing methods typically make limiting distributional assumptions, modeling assumptions, and require data splitting. In this work, we develop a novel, mostly model-agnostic, and distribution-free inference framework for feature importance in regression or classification tasks that does not require data splitting. Our approach leverages a form of random observation and feature subsampling called minipatch ensembles; it utilizes the trained ensembles for inference and requires no model-refitting or held-out test data after training. We show that our approach enjoys both computational and statistical efficiency as well as circumvents interpretational challenges with data splitting. Further, despite using the same data for training and inference, we show the asymptotic validity of our confidence intervals under mild assumptions. Additionally, we propose theory-supported solutions to critical practical issues including vanishing variance for null features and inference after data-driven tuning for hyperparameters. We demonstrate the advantages of our approach over existing methods on a series of synthetic and real data examples.

翻译：特征重要性推断对于机器学习模型的可解释性和可靠性至关重要。近年来，开发模型无关方法来解释任意预测模型（通常采用特征遮挡或留一协变量排除（LOCO）推断形式）的研究日益受到关注。现有方法通常需要假设分布限制、模型假设，并要求进行数据分割。在本工作中，我们提出了一种新颖的、基本模型无关且无分布假设的推断框架，适用于回归或分类任务中的特征重要性推断，且无需数据分割。该方法利用一种称为最小补丁集成的随机观测和特征子采样形式，通过训练后的集成模型进行推断，无需重新拟合模型或在训练后保留测试数据。研究表明，我们的方法兼具计算效率和统计效能，同时规避了数据分割带来的可解释性挑战。此外，尽管训练和推断使用相同数据，我们证明在温和假设下置信区间具有渐近有效性。我们还针对关键实际问题提出了理论支持的解决方案，包括空特征的方差消失问题以及超参数数据驱动调优后的推断问题。通过一系列合成数据和真实数据实验，我们证明了该方法相较于现有方法的优势。