The highly irregular spiking activity of cortical neurons and behavioral variability suggest that the brain could operate in a fundamentally probabilistic way. Mimicking how the brain implements and learns probabilistic computation could be a key to developing machine intelligence that can think more like humans. In this work, we propose a theory of stochastic neural computing (SNC) in which streams of noisy inputs are transformed and processed through populations of nonlinearly coupled spiking neurons. To account for the propagation of correlated neural variability, we derive from first principles a moment embedding for spiking neural network (SNN). This leads to a new class of deep learning model called the moment neural network (MNN) which naturally generalizes rate-based neural networks to second order. As the MNN faithfully captures the stationary statistics of spiking neural activity, it can serve as a powerful proxy for training SNN with zero free parameters. Through joint manipulation of mean firing rate and noise correlations in a task-driven way, the model is able to learn inference tasks while simultaneously minimizing prediction uncertainty, resulting in enhanced inference speed. We further demonstrate the application of our method to Intel's Loihi neuromorphic hardware. The proposed theory of SNC may open up new opportunities for developing machine intelligence capable of computing uncertainty and for designing unconventional computing architectures.

翻译：皮层神经元高度不规则的动作电位活动以及行为变异提示，大脑可能以基本概率化的方式运作。模仿大脑实现和学习概率计算，或许是开发更接近人类思维方式的机器智能的关键。本研究提出随机神经计算（SNC）理论，其中噪声输入流通过非线性耦合的神经元群体进行转换与处理。为解释相关神经变异性的传播，我们基于第一性原理推导出脉冲神经网络（SNN）的矩嵌入方法。由此衍生出一类新型深度学习模型——矩神经网络（MNN），该模型将基于速率的神经网络自然推广至二阶。由于MNN可精确捕捉脉冲神经活动的稳态统计特性，它可作为零自由参数训练SNN的有力代理。通过对平均发放率和噪声相关性进行任务驱动的联合调控，该模型能在学习推理任务的同时最小化预测不确定性，从而提升推理速度。我们进一步验证了该方法在英特尔Loihi神经形态硬件上的应用。所提出的SNC理论有望为开发具有不确定性计算能力的机器智能，以及设计非常规计算架构开辟新途径。