In sampling-based Bayesian models of brain function, neural activities are assumed to be samples from probability distributions that the brain uses for probabilistic computation. However, a comprehensive understanding of how mechanistic models of neural dynamics can sample from arbitrary distributions is still lacking. We use tools from functional analysis and stochastic differential equations to explore the minimum architectural requirements for $\textit{recurrent}$ neural circuits to sample from complex distributions. We first consider the traditional sampling model consisting of a network of neurons whose outputs directly represent the samples (sampler-only network). We argue that synaptic current and firing-rate dynamics in the traditional model have limited capacity to sample from a complex probability distribution. We show that the firing rate dynamics of a recurrent neural circuit with a separate set of output units can sample from an arbitrary probability distribution. We call such circuits reservoir-sampler networks (RSNs). We propose an efficient training procedure based on denoising score matching that finds recurrent and output weights such that the RSN implements Langevin sampling. We empirically demonstrate our model's ability to sample from several complex data distributions using the proposed neural dynamics and discuss its applicability to developing the next generation of sampling-based brain models.

翻译：在基于采样的脑功能贝叶斯模型中，神经活动被假定为大脑用于概率计算的概率分布样本。然而，对于神经动力学的机械模型如何从任意分布中进行采样，我们仍缺乏全面理解。我们利用泛函分析和随机微分方程的工具，探究循环神经回路从复杂分布中采样所需的最小结构要求。首先考虑传统采样模型，该模型由输出直接代表样本的神经元网络组成（仅采样器网络）。我们认为传统模型中突触电流和发放率动力学的采样能力有限，无法从复杂概率分布中有效采样。我们证明，具有独立输出单元集合的循环神经回路，其发放率动力学能够从任意概率分布中采样。我们将此类回路称为储层采样器网络（RSN）。基于去噪得分匹配，我们提出一种高效训练方法，可确定使RSN实现朗之万采样的循环权重和输出权重。我们通过实验证明了所提模型利用该神经动力学从多个复杂数据分布中采样的能力，并讨论了其在开发下一代基于采样的脑模型中的适用性。