We establish estimations for the parameters of the output distribution for the softmax activation function using the probit function. As an application, we develop a new efficient Bayesian learning algorithm for fully connected neural networks, where training and predictions are performed within the Bayesian inference framework in closed-form. This approach allows sequential learning and requires no computationally expensive gradient calculation and Monte Carlo sampling. Our work generalizes the Bayesian algorithm for a single perceptron for binary classification in \cite{H} to multi-layer perceptrons for multi-class classification.

翻译：我们利用probit函数建立了softmax激活函数输出分布参数的估计方法。作为应用，我们开发了一种针对全连接神经网络的高效贝叶斯学习算法，其中训练和预测均在贝叶斯推理框架内以闭合形式完成。该方法支持顺序学习，且无需计算昂贵的梯度计算和蒙特卡洛采样。我们的工作将文献\cite{H}中用于二分类单感知机的贝叶斯算法推广至用于多分类的多层感知机。