Kernel-free quadratic surface support vector machines (QSVM) have recently gained traction due to their flexibility in modeling nonlinear decision boundaries without relying on kernel functions. However, the introduction of a full quadratic classifier significantly increases the number of model parameters, scaling quadratically with data dimensionality, which often leads to overfitting and makes interpretation difficult. To address these challenges, we propose sparse variants of the QSVM by enforcing a cardinality constraint on the model parameters. While enhancing generalization and promoting sparsity, leveraging the $\ell_0$-norm inevitably incurs additional computational complexity. To tackle this, we develop a penalty decomposition algorithm capable of producing solutions that provably satisfy the first-order Lu-Zhang optimality conditions. We show that the subproblems arising within the algorithm either admit closed-form solutions or can be solved efficiently through dual formulations, which contributes to the method's overall effectiveness. Besides, we analyze the convergence behavior of the algorithm under both loss settings. In addition, the numerical experiments on public benchmark datasets indicate that the proposed model is competitive with commonly used SVM variants and produces sparse solutions as expected. Moreover, its strong performance on real-world credit datasets demonstrates its potential for credit scoring applications.

翻译：无核二次曲面支持向量机（QSVM）因其无需依赖核函数即可灵活建模非线性决策边界的特点，近年来受到广泛关注。然而，完整二次分类器的引入显著增加了模型参数的数量，其规模随数据维度呈二次增长，这往往导致过拟合并使模型解释变得困难。为应对这些挑战，我们通过对模型参数施加基数约束，提出了QSVM的稀疏变体。在提升泛化能力与促进稀疏性的同时，利用$\ell_0$范数不可避免地会引入额外的计算复杂度。为此，我们开发了一种惩罚分解算法，该算法能够生成严格满足Lu-Zhang一阶最优性条件的解。我们证明算法中产生的子问题要么存在闭式解，要么可通过对偶形式高效求解，这确保了方法的整体有效性。此外，我们分析了算法在两种损失设置下的收敛行为。在公开基准数据集上的数值实验表明，所提模型与常用SVM变体具有竞争力，并能按预期生成稀疏解。同时，该模型在真实信用数据集上的优异表现，证明了其在信用评分应用中的潜力。