Building robust, interpretable, and secure AI system requires quantifying and representing uncertainty under a probabilistic perspective to mimic human cognitive abilities. However, probabilistic computation presents significant challenges for most conventional artificial neural network, as they are essentially implemented in a deterministic manner. In this paper, we develop an efficient probabilistic computation framework by truncating the probabilistic representation of neural activation up to its mean and covariance and construct a moment neural network that encapsulates the nonlinear coupling between the mean and covariance of the underlying stochastic network. We reveal that when only the mean but not the covariance is supervised during gradient-based learning, the unsupervised covariance spontaneously emerges from its nonlinear coupling with the mean and faithfully captures the uncertainty associated with model predictions. Our findings highlight the inherent simplicity of probabilistic computation by seamlessly incorporating uncertainty into model prediction, paving the way for integrating it into large-scale AI systems.

翻译：构建鲁棒、可解释且安全的人工智能系统，需要从概率视角量化与表征不确定性以模拟人类认知能力。然而，概率计算对大多数传统人工神经网络构成显著挑战——这些网络本质上以确定性方式实现。本文通过将神经激活的概率表征截断至均值与协方差层面，发展出高效概率计算框架，构建了蕴含底层随机网络均值与协方差非线性耦合的矩神经网络。我们发现，当基于梯度的学习过程仅监督均值而未监督协方差时，无监督协方差会自发从其与均值的非线性耦合中涌现，并忠实捕捉模型预测相关的不确定性。本研究成果揭示了概率计算的内在简洁性——将不确定性无缝融入模型预测，为将其集成至大规模人工智能系统铺平道路。