Kernel ridge regression, KRR, is a generalization of linear ridge regression that is non-linear in the data, but linear in the model parameters. Here, we introduce an equivalent formulation of the objective function of KRR, which opens up for replacing the ridge penalty with the $\ell_\infty$ and $\ell_1$ penalties. Using the $\ell_\infty$ and $\ell_1$ penalties, we obtain robust and sparse kernel regression, respectively. We study the similarities between explicitly regularized kernel regression and the solutions obtained by early stopping of iterative gradient-based methods, where we connect $\ell_\infty$ regularization to sign gradient descent, $\ell_1$ regularization to forward stagewise regression (also known as coordinate descent), and $\ell_2$ regularization to gradient descent, and, in the last case, theoretically bound for the differences. We exploit the close relations between $\ell_\infty$ regularization and sign gradient descent, and between $\ell_1$ regularization and coordinate descent to propose computationally efficient methods for robust and sparse kernel regression. We finally compare robust kernel regression through sign gradient descent to existing methods for robust kernel regression on five real data sets, demonstrating that our method is one to two orders of magnitude faster, without compromised accuracy.

翻译：核岭回归（Kernel Ridge Regression，KRR）是线性岭回归的一种推广形式，它在数据上呈现非线性，但在模型参数上保持线性。本文提出了KRR目标函数的一种等价表述，从而使得用$\ell_\infty$和$\ell_1$惩罚项替代岭惩罚成为可能。使用$\ell_\infty$惩罚项可获得鲁棒的核回归，而使用$\ell_1$惩罚项则可获得稀疏的核回归。我们研究了显式正则化核回归与基于梯度的迭代方法早停所得解之间的相似性，其中将$\ell_\infty$正则化与符号梯度下降、$\ell_1$正则化与前向分段回归（亦称坐标下降）、$\ell_2$正则化与梯度下降分别建立联系，并在最后一种情形下从理论上界定了其差异。我们利用$\ell_\infty$正则化与符号梯度下降之间、以及$\ell_1$正则化与坐标下降之间的紧密联系，提出了计算高效的鲁棒与稀疏核回归方法。最后，我们在五个真实数据集上，将通过符号梯度下降实现的鲁棒核回归与现有鲁棒核回归方法进行比较，结果表明我们的方法在保持精度不受损的前提下，计算速度提升了一到两个数量级。