Feature Selection is a crucial procedure in Data Science tasks such as Classification, since it identifies the relevant variables, making thus the classification procedures more interpretable, cheaper in terms of measurement and more effective by reducing noise and data overfit. The relevance of features in a classification procedure is linked to the fact that misclassifications costs are frequently asymmetric, since false positive and false negative cases may have very different consequences. However, off-the-shelf Feature Selection procedures seldom take into account such cost-sensitivity of errors. In this paper we propose a mathematical-optimization-based Feature Selection procedure embedded in one of the most popular classification procedures, namely, Support Vector Machines, accommodating asymmetric misclassification costs. The key idea is to replace the traditional margin maximization by minimizing the number of features selected, but imposing upper bounds on the false positive and negative rates. The problem is written as an integer linear problem plus a quadratic convex problem for Support Vector Machines with both linear and radial kernels. The reported numerical experience demonstrates the usefulness of the proposed Feature Selection procedure. Indeed, our results on benchmark data sets show that a substantial decrease of the number of features is obtained, whilst the desired trade-off between false positive and false negative rates is achieved.

翻译：特征选择是分类等数据科学任务中的关键步骤，因为它能够识别相关变量，从而通过降低噪声和数据过拟合，使分类过程更具可解释性、测量成本更低且更有效。特征在分类过程中的重要性往往与误分类成本的不对称性相关，因为假正例和假负例的后果可能差异显著。然而，现成的特征选择方法很少考虑这种错误的代价敏感性。在本文中，我们提出了一种基于数学优化的特征选择方法，将其嵌入最流行的分类方法之一——支持向量机中，并纳入不对称的误分类成本。其核心思想是：通过最小化所选特征数量来替代传统的间隔最大化，同时为假正例率和假负例率设定上限。该问题被表述为一个整数线性规划问题，并结合一个适用于线性核与径向基核的支持向量机二次凸优化问题。报告的数值实验证明了本文提出的特征选择方法的有效性。事实上，我们在基准数据集上的结果表明，该方法在实现假正例率和假负例率之间的期望权衡的同时，能够显著减少所选特征的数量。