In the context of deep learning with kernel machines, the deep Restricted Kernel Machine (DRKM) framework allows multiple levels of kernel PCA (KPCA) and Least-Squares Support Vector Machines (LSSVM) to be combined into a deep architecture using visible and hidden units. We propose a new method for DRKM classification coupling the objectives of KPCA and classification levels, with the hidden feature matrix lying on the Stiefel manifold. The classification level can be formulated as an LSSVM or as an MLP feature map, combining depth in terms of levels and layers. The classification level is expressed in its primal formulation, as the deep KPCA levels, in their dual formulation, can embed the most informative components of the data in a much lower dimensional space. The dual setting is independent of the dimension of the inputs and the primal setting is parametric, which makes the proposed method computationally efficient for both high-dimensional inputs and large datasets. In the experiments, we show that our developed algorithm can effectively learn from small datasets, while using less memory than the convolutional neural network (CNN) with high-dimensional data. and that models with multiple KPCA levels can outperform models with a single level. On the tested larger-scale datasets, DRKM is more energy efficient than CNN while maintaining comparable performance.

翻译：在核机深度学习的背景下，深层限制核机（DRKM）框架允许将多个层次的核主成分分析（KPCA）和最小二乘支持向量机（LSSVM）通过可见单元与隐藏单元组合成深度架构。我们提出了一种新的DRKM分类方法，该方法将KPCA目标与分类层级的目标相耦合，其中隐藏特征矩阵位于施蒂费尔流形上。分类层级可表述为LSSVM或MLP特征映射，从而融合了层级深度与网络层深度。分类层级采用原对偶形式表达，而深层KPCA层级通过其对偶形式能将数据中最具信息量的成分嵌入到维度显著降低的空间中。对偶设置独立于输入维度，原设置则具有参数化特性，这使得所提方法在处理高维输入和大规模数据集时均具有计算高效性。实验表明，我们开发的算法能有效从小规模数据集中学习，同时在使用高维数据时比卷积神经网络（CNN）消耗更少内存；且包含多个KPCA层级的模型性能优于单层级模型。在测试的较大规模数据集上，DRKM在保持与CNN相当性能的同时具有更高的能效。