For regression tasks, standard Gaussian processes (GPs) provide natural uncertainty quantification (UQ), while deep neural networks (DNNs) excel at representation learning. Deterministic UQ methods for neural networks have successfully combined the two and require only a single pass through the neural network. However, current methods necessitate changes to network training to address feature collapse, where unique inputs map to identical feature vectors. We propose an alternative solution, the deep Vecchia ensemble (DVE), which allows deterministic UQ to work in the presence of feature collapse, negating the need for network retraining. DVE comprises an ensemble of GPs built on hidden-layer outputs of a DNN, achieving scalability via Vecchia approximations that leverage nearest-neighbor conditional independence. DVE is compatible with pretrained networks and incurs low computational overhead. We demonstrate DVE's utility on several datasets and carry out experiments to understand the inner workings of the proposed method.

翻译：对于回归任务，标准高斯过程（GPs）可提供天然的不确定性量化（UQ），而深度神经网络（DNNs）在表示学习方面表现卓越。针对神经网络的确定性UQ方法已成功将两者结合，且仅需对神经网络进行一次前向传播。然而，现有方法需要调整网络训练以解决特征坍缩问题——即不同输入映射至相同特征向量的现象。我们提出一种替代解决方案：深度Vecchia集成（DVE），该方法使确定性UQ能够在特征坍缩存在时有效工作，从而无需重新训练网络。DVE由构建在DNN隐藏层输出上的GP集成构成，通过利用最近邻条件独立性的Vecchia近似实现可扩展性。DVE兼容预训练网络且计算开销较低。我们在多个数据集上验证了DVE的实用性，并通过实验深入解析了所提方法的内在机理。