Feature selection is a classical problem in statistics and machine learning, and it continues to remain an extremely challenging problem especially in the context of unknown non-linear relationships with dependent features. On the other hand, Shapley values are a classic solution concept from cooperative game theory that is widely used for feature attribution in general non-linear models with highly-dependent features. However, Shapley values are not naturally suited for feature selection since they tend to capture both direct effects from each feature to the response and indirect effects through other features. In this paper, we combine the advantages of Shapley values and adapt them to feature selection by proposing \emph{MinShap}, a modification of the Shapley value framework along with a suite of other related algorithms. In particular for MinShap, instead of taking the average marginal contributions over permutations of features, considers the minimum marginal contribution across permutations. We provide a theoretical foundation motivated by the faithfulness assumption in DAG (directed acyclic graphical models), a guarantee for the Type I error of MinShap, and show through numerical simulations and real data experiments that MinShap tends to outperform state-of-the-art feature selection algorithms such as LOCO, GCM and Lasso in terms of both accuracy and stability. We also introduce a suite of algorithms related to MinShap by using the multiple testing/p-value perspective that improves performance in lower-sample settings and provide supporting theoretical guarantees.

翻译：特征选择是统计学与机器学习中的经典问题，尤其在面对具有依赖性的未知非线性关系时仍极具挑战性。另一方面，沙普利值作为合作博弈论中的经典解概念，被广泛用于高度依赖特征的一般非线性模型中的特征归因。然而，沙普利值并不天然适用于特征选择，因其倾向于同时捕捉每个特征对响应的直接效应以及通过其他特征产生的间接效应。本文结合沙普利值的优势并对其进行适配，提出MinShap——一个对沙普利值框架的改良及其配套算法系列。具体而言，MinShap不再计算特征排列上的平均边际贡献，而是考虑各排列中的最小边际贡献。我们基于DAG（有向无环图模型）中的忠实性假设提供了理论依据，保证了MinShap的第一类错误率，并通过数值模拟和真实数据实验证明，MinShap在准确性和稳定性方面均优于LOCO、GCM和Lasso等前沿特征选择算法。我们还从多重检验/显著性水平角度引入与MinShap相关的算法系列，以提升小样本场景下的性能，并提供了相应的理论保障。