Implicit processes (IPs) are a generalization of Gaussian processes (GPs). IPs may lack a closed-form expression but are easy to sample from. Examples include, among others, Bayesian neural networks or neural samplers. IPs can be used as priors over functions, resulting in flexible models with well-calibrated prediction uncertainty estimates. Methods based on IPs usually carry out function-space approximate inference, which overcomes some of the difficulties of parameter-space approximate inference. Nevertheless, the approximations employed often limit the expressiveness of the final model, resulting, e.g., in a Gaussian predictive distribution, which can be restrictive. We propose here a multi-layer generalization of IPs called the Deep Variational Implicit process (DVIP). This generalization is similar to that of deep GPs over GPs, but it is more flexible due to the use of IPs as the prior distribution over the latent functions. We describe a scalable variational inference algorithm for training DVIP and show that it outperforms previous IP-based methods and also deep GPs. We support these claims via extensive regression and classification experiments. We also evaluate DVIP on large datasets with up to several million data instances to illustrate its good scalability and performance.

翻译：隐式过程（IPs）是高斯过程（GPs）的一种推广。隐式过程可能缺乏闭式表达式，但易于采样，例如贝叶斯神经网络或神经采样器等。隐式过程可作为函数上的先验分布，从而得到具有良好校准预测不确定性估计的灵活模型。基于隐式过程的方法通常进行函数空间近似推理，这克服了参数空间近似推理的某些困难。然而，所采用的近似常常限制了最终模型的表达能力，例如产生高斯预测分布，这可能导致限制性。本文提出一种称为深度变分隐式过程（DVIP）的多层隐式过程推广。这种推广类似于深层高斯过程对高斯过程的推广，但由于使用隐式过程作为隐函数的先验分布而更具灵活性。我们描述了可扩展的变分推理算法来训练DVIP，并证明其优于先前基于隐式过程的方法以及深层高斯过程。通过大量回归和分类实验支撑了这些主张。我们还在多达数百万数据实例的大规模数据集上评估DVIP，以说明其良好的可扩展性和性能。