The application of neural network models to scientific machine learning tasks has proliferated in recent years. In particular, neural network models have proved to be adept at modeling processes with spatial-temporal complexity. Nevertheless, these highly parameterized models have garnered skepticism in their ability to produce outputs with quantified error bounds over the regimes of interest. Hence there is a need to find uncertainty quantification methods that are suitable for neural networks. In this work we present comparisons of the parametric uncertainty quantification of neural networks modeling complex spatial-temporal processes with Hamiltonian Monte Carlo and Stein variational gradient descent and its projected variant. Specifically we apply these methods to graph convolutional neural network models of evolving systems modeled with recurrent neural network and neural ordinary differential equations architectures. We show that Stein variational inference is a viable alternative to Monte Carlo methods with some clear advantages for complex neural network models. For our exemplars, Stein variational interference gave similar uncertainty profiles through time compared to Hamiltonian Monte Carlo, albeit with generally more generous variance.Projected Stein variational gradient descent also produced similar uncertainty profiles to the non-projected counterpart, but large reductions in the active weight space were confounded by the stability of the neural network predictions and the convoluted likelihood landscape.

翻译：近年来，神经网络模型在科学机器学习任务中的应用日益广泛。特别地，神经网络模型已被证明擅长模拟具有时空复杂性的过程。然而，这些高参数化模型在感兴趣的场景下能否输出具有量化误差界的结果仍受到质疑。因此，亟需寻找适用于神经网络的不确定性量化方法。本研究采用哈密顿蒙特卡洛方法、斯坦因变分梯度下降及其投影变体，对模拟复杂时空过程的神经网络参数化不确定性量化进行了比较分析。具体而言，我们将这些方法应用于由循环神经网络和神经常微分方程架构建模的演化系统图卷积神经网络模型。研究表明，斯坦因变分推断可作为蒙特卡洛方法的可行替代方案，在处理复杂神经网络模型时具有明显优势。在示例中，斯坦因变分推断与哈密顿蒙特卡洛方法在时间维度上产生了相似的不确定性分布，尽管其方差通常更大。投影斯坦因变分梯度下降方法也产生了与非投影方法类似的不确定性分布，但神经网络预测的稳定性与复杂似然面导致活跃权重空间的大幅缩减出现混杂效应。