A simple and intuitive method for feature selection consists of choosing the feature subset that maximizes a nonparametric measure of dependence between the response and the features. A popular proposal from the literature uses the Hilbert-Schmidt Independence Criterion (HSIC) as the nonparametric dependence measure. The rationale behind this approach to feature selection is that important features will exhibit a high dependence with the response and their inclusion in the set of selected features will increase the HSIC. Through counterexamples, we demonstrate that this rationale is flawed and that feature selection via HSIC maximization can miss critical features.

翻译：一种简单直观的特征选择方法在于选择能够最大化响应变量与特征之间非参数依赖度量的特征子集。文献中一种流行的方案采用希尔伯特-施密特独立性准则（HSIC）作为非参数依赖度量。这种特征选择方法的基本原理是：重要特征会表现出与响应变量的高度依赖性，将其纳入所选特征集会提升HSIC值。通过反例论证，我们证明该原理存在缺陷，基于HSIC最大化的特征选择可能会遗漏关键特征。