In this work, we use Deep Gaussian Processes (DGPs) as statistical surrogates for stochastic processes with complex distributions. Conventional inferential methods for DGP models can suffer from high computational complexity as they require large-scale operations with kernel matrices for training and inference. In this work, we propose an efficient scheme for accurate inference and efficient training based on a range of Gaussian Processes, called the Tensor Markov Gaussian Processes (TMGP). We construct an induced approximation of TMGP referred to as the hierarchical expansion. Next, we develop a deep TMGP (DTMGP) model as the composition of multiple hierarchical expansion of TMGPs. The proposed DTMGP model has the following properties: (1) the outputs of each activation function are deterministic while the weights are chosen independently from standard Gaussian distribution; (2) in training or prediction, only polylog(M) (out of M) activation functions have non-zero outputs, which significantly boosts the computational efficiency. Our numerical experiments on synthetic models and real datasets show the superior computational efficiency of DTMGP over existing DGP models.

翻译：本文利用深度高斯过程作为具有复杂分布的随机过程的统计替代模型。传统的深度高斯过程模型推理方法因需要处理核矩阵的大规模运算，在训练和推理过程中面临高计算复杂度的挑战。为此，我们提出了一种基于高斯过程族的高效推断与训练方案，即张量马尔可夫高斯过程。我们构建了该过程的诱导近似方法，称为层次扩展。进一步，我们开发了深度融合张量马尔可夫高斯过程模型，该模型由多个张量马尔可夫高斯过程的层次扩展复合而成。所提出的模型具有以下特性：（1）每层激活函数的输出是确定性的，而权重从标准高斯分布中独立采样；（2）在训练或预测时，仅有 polylog(M) 个（M个中）激活函数产生非零输出，从而显著提升计算效率。在合成模型与真实数据集上的数值实验表明，该模型相比现有深度高斯过程模型具有更优的计算效率。