Uncertainty quantification in neural networks prediction is a main issue for usual applications. Our approach seeks at reducing computation costs by directly evaluating uncertainty using PDE's information on the asymptotic variance, rather than the deep ensemble method which may be seen as a Monte Carlo estimation of the prediction, requiring the training of multiple networks. We thus study the law of the limiting process describing the random fluctuations around the mean-field limit of wide two-layer neural networks trained by stochastic gradient descent in a weak-noise regime. Building on a recent trajectorial central limit theorem, in which this limit is characterized as the weak solution of a linear stochastic evolution equation, we identify its law explicitly. More precisely, we show that it is a centered Gaussian process in the dual of a weighted Sobolev space, and we derive a closed covariance representation for the finite-dimensional distributions obtained by testing it against smooth functions. This covariance is expressed through the solution of a backward transport equation with a nonlocal source term, whose coefficients are driven by the mean-field trajectory. As a consequence, by testing against the activation function at a fixed input, we obtain an expression for the limiting variance of the corresponding network-output fluctuations. We illustrate this result numerically on a one-dimensional regression example.

翻译：神经网络预测中的不确定性量化是实际应用中的核心问题。我们的方法旨在通过利用偏微分方程中关于渐近方差的信息直接评估不确定性，而非采用需要训练多个网络的深度集成方法（该方法可视为预测的蒙特卡洛估计），从而降低计算成本。为此，我们研究了弱噪声机制下随机梯度下降训练的宽双层神经网络在平均场极限附近随机波动的极限过程律。基于近期针对该极限的轨迹中心极限定理（该定理将其刻画为线性随机演化方程的弱解），我们显式地确定了其分布律。更精确地说，我们证明该过程是加权Sobolev空间对偶空间中的中心高斯过程，并推导出其与光滑函数测试所得有限维分布的闭式协方差表示。该协方差通过含非局部源项的后向迁移方程解表示，其系数由平均场轨迹驱动。由此，通过固定输入处激活函数的测试，我们获得了相应网络输出波动的极限方差表达式。我们通过一维回归算例对该结果进行数值验证。