Despite their many desirable properties, Gaussian processes (GPs) are often compared unfavorably to deep neural networks (NNs) for lacking the ability to learn representations. Recent efforts to bridge the gap between GPs and deep NNs have yielded a new class of inter-domain variational GPs in which the inducing variables correspond to hidden units of a feedforward NN. In this work, we examine some practical issues associated with this approach and propose an extension that leverages the orthogonal decomposition of GPs to mitigate these limitations. In particular, we introduce spherical inter-domain features to construct more flexible data-dependent basis functions for both the principal and orthogonal components of the GP approximation and show that incorporating NN activation features under this framework not only alleviates these shortcomings but is more scalable than alternative strategies. Experiments on multiple benchmark datasets demonstrate the effectiveness of our approach.

翻译：尽管高斯过程（GPs）具有许多理想特性，但由于缺乏学习表征的能力，它们常被与深度神经网络（NNs）进行不利比较。近期弥合高斯过程与深度神经网络之间差距的努力催生了一类新的跨域变分高斯过程，其中诱导变量对应于前馈神经网络的隐藏单元。本文研究了该方法相关的一些实际问题，并提出一种扩展方案，通过利用高斯过程的正交分解来缓解这些局限性。具体而言，我们引入球形跨域特征来构建更灵活的数据相关基函数，用于高斯过程近似的主成分和正交成分，并证明在该框架下融入神经网络激活特征不仅能缓解这些缺陷，而且比替代策略更具可扩展性。在多个基准数据集上的实验验证了我们方法的有效性。