The accuracy and complexity of machine learning algorithms based on kernel optimization are determined by the set of kernels over which they are able to optimize. An ideal set of kernels should: admit a linear parameterization (for tractability); be dense in the set of all kernels (for robustness); be universal (for accuracy). Recently, a framework was proposed for using positive matrices to parameterize a class of positive semi-separable kernels. Although this class can be shown to meet all three criteria, previous algorithms for optimization of such kernels were limited to classification and furthermore relied on computationally complex Semidefinite Programming (SDP) algorithms. In this paper, we pose the problem of learning semiseparable kernels as a minimax optimization problem and propose a SVD-QCQP primal-dual algorithm which dramatically reduces the computational complexity as compared with previous SDP-based approaches. Furthermore, we provide an efficient implementation of this algorithm for both classification and regression -- an implementation which enables us to solve problems with 100 features and up to 30,000 datums. Finally, when applied to benchmark data, the algorithm demonstrates the potential for significant improvement in accuracy over typical (but non-convex) approaches such as Neural Nets and Random Forest with similar or better computation time.

翻译：基于核优化的机器学习算法的精度与复杂度取决于其能够优化的核函数集合。理想的核函数集合应满足：允许线性参数化（以便于处理）、在全体核函数空间中稠密（以确保鲁棒性）、且具有通用性（以保证精度）。近期，研究者提出利用正定矩阵对一类正半可分离核函数进行参数化的框架。尽管该核类已证明满足上述三项标准，但此前针对此类核的优化算法仅适用于分类问题，且依赖计算复杂的半定规划（SDP）算法。本文将半可分离核的学习问题转化为极小极大优化问题，并提出一种SVD-QCQP原始-对偶算法，其计算复杂度相较基于SDP的方法显著降低。此外，我们为分类与回归任务提供了该算法的高效实现——该实现可求解包含100个特征与多达30,000个样本的问题。在基准数据集上的实验表明，该算法在计算时间与现有方法相当或更优的条件下，相比典型（但非凸）方法（如神经网络与随机森林）展现出显著提升精度的潜力。