We study Sparse Multiple Kernel Learning (SMKL), which is the problem of selecting a sparse convex combination of prespecified kernels for support vector binary classification. Unlike prevailing l1 regularized approaches that approximate a sparsifying penalty, we formulate the problem by imposing an explicit cardinality constraint on the kernel weights and add an l2 penalty for robustness. We solve the resulting non-convex minimax problem via an alternating best response algorithm with two subproblems: the alpha subproblem is a standard kernel SVM dual solved via LIBSVM, while the beta subproblem admits an efficient solution via the Greedy Selector and Simplex Projector algorithm. We reformulate SMKL as a mixed integer semidefinite optimization problem and derive a hierarchy of semidefinite convex relaxations which can be used to certify near-optimality of the solutions returned by our best response algorithm and also to warm start it. On ten UCI benchmarks, our method with random initialization outperforms state-of-the-art MKL approaches in out-of-sample prediction accuracy on average by 3.34 percentage points (relative to the best performing benchmark) while selecting a small number of candidate kernels in comparable runtime. With warm starting, our method outperforms the best performing benchmark's out-of-sample prediction accuracy on average by 4.05 percentage points. Our convex relaxations provide a certificate that in several cases, the solution returned by our best response algorithm is the globally optimal solution.

翻译：本文研究稀疏多核学习问题，即针对支持向量机二分类任务，从预设核函数集合中选择一个稀疏凸组合。不同于主流采用l1正则化近似稀疏惩罚的方法，我们通过显式施加核权重基数约束并添加l2正则项以增强鲁棒性来构建问题模型。我们通过交替最优响应算法求解所得非凸极小极大问题，该算法包含两个子问题：alpha子问题为标准核支持向量机对偶问题，可通过LIBSVM求解；beta子问题则可通过贪婪选择器与单纯形投影器算法高效求解。我们将稀疏多核学习重构为混合整数半定优化问题，并推导出一系列半定凸松弛层级，这些松弛既可用于验证交替最优响应算法所得解的近似最优性，也可用于对算法进行热启动。在十个UCI基准数据集上的实验表明，随机初始化的方法在可比运行时间内选择少量候选核的同时，其样本外预测准确率平均优于当前最优多核学习方法3.34个百分点（相对于最佳基准方法）。采用热启动后，我们的方法平均优于最佳基准方法样本外预测准确率4.05个百分点。我们的凸松弛方法提供了可验证证书，在多个案例中证明交替最优响应算法返回的解即为全局最优解。