In contrast to deep networks, kernel methods cannot directly take advantage of depth. In this regard, the deep Restricted Kernel Machine (DRKM) framework allows multiple levels of kernel PCA (KPCA) and Least-Squares Support Vector Machines (LSSVM) to be combined into a deep architecture using visible and hidden units. We propose a new method for DRKM classification coupling the objectives of KPCA and classification levels, with the hidden feature matrix lying on the Stiefel manifold. The classification level can be formulated as an LSSVM or as an MLP feature map, combining depth in terms of levels and layers. The classification level is expressed in its primal formulation, as the deep KPCA levels can embed the most informative components of the data in a much lower dimensional space. In the experiments on benchmark datasets with few available training points, we show that our deep method improves over the LSSVM/MLP and that models with multiple KPCA levels can outperform models with a single level.

翻译：与深度网络不同，核方法无法直接利用深度结构的优势。为此，深度受限核机（DRKM）框架通过可见单元和隐单元，将多层核主成分分析（KPCA）和最小二乘支持向量机（LSSVM）整合为深度架构。我们提出了一种新的DRKM分类方法，该方法将KPCA与分类层目标耦合，且隐藏特征矩阵位于Stiefel流形上。分类层可表述为LSSVM或MLP特征映射，从而在层级和层数两个维度上结合深度。分类层采用原始形式表达，而深层KPCA层能将数据中最具信息量的成分嵌入到维数显著降低的空间中。在可用训练样本较少的基准数据集实验中，我们证明所提出的深度方法优于LSSVM/MLP，且多KPCA层模型可优于单层模型。