By leveraging the kernel trick in the output space, kernel-induced losses provide a principled way to define structured output prediction tasks for a wide variety of output modalities. In particular, they have been successfully used in the context of surrogate non-parametric regression, where the kernel trick is typically exploited in the input space as well. However, when inputs are images or texts, more expressive models such as deep neural networks seem more suited than non-parametric methods. In this work, we tackle the question of how to train neural networks to solve structured output prediction tasks, while still benefiting from the versatility and relevance of kernel-induced losses. We design a novel family of deep neural architectures, whose last layer predicts in a data-dependent finite-dimensional subspace of the infinite-dimensional output feature space deriving from the kernel-induced loss. This subspace is chosen as the span of the eigenfunctions of a randomly-approximated version of the empirical kernel covariance operator. Interestingly, this approach unlocks the use of gradient descent algorithms (and consequently of any neural architecture) for structured prediction. Experiments on synthetic tasks as well as real-world supervised graph prediction problems show the relevance of our method.

翻译：通过在输出空间利用核技巧，核诱导损失为多种输出模态的结构化输出预测任务提供了原则性的定义方式。特别是在代理非参数回归的背景下，核技巧通常在输入空间同样得到利用。然而，当输入为图像或文本时，深度神经网络等更具表达能力的模型似乎比非参数方法更为适用。本研究旨在探讨如何训练神经网络以解决结构化输出预测任务，同时仍能受益于核诱导损失的通用性与相关性。我们设计了一个新颖的深度神经架构家族，其最后一层在由核诱导损失衍生的无限维输出特征空间的数据依赖有限维子空间中进行预测。该子空间被选为经验核协方差算子随机近似版本的特征函数张成的空间。值得注意的是，此方法解锁了梯度下降算法（以及随之而来的任何神经架构）在结构化预测中的应用。在合成任务及现实世界监督图预测问题上的实验验证了我们方法的有效性。