In sampling-based Bayesian models of brain function, neural activities are assumed to be samples from probability distributions that the brain uses for probabilistic computation. However, a comprehensive understanding of how mechanistic models of neural dynamics can sample from arbitrary distributions is still lacking. We use tools from functional analysis and stochastic differential equations to explore the minimum architectural requirements for $\textit{recurrent}$ neural circuits to sample from complex distributions. We first consider the traditional sampling model consisting of a network of neurons whose outputs directly represent the samples (sampler-only network). We argue that synaptic current and firing-rate dynamics in the traditional model have limited capacity to sample from a complex probability distribution. We show that the firing rate dynamics of a recurrent neural circuit with a separate set of output units can sample from an arbitrary probability distribution. We call such circuits reservoir-sampler networks (RSNs). We propose an efficient training procedure based on denoising score matching that finds recurrent and output weights such that the RSN implements Langevin sampling. We empirically demonstrate our model's ability to sample from several complex data distributions using the proposed neural dynamics and discuss its applicability to developing the next generation of sampling-based brain models.

翻译：在基于采样的脑功能贝叶斯模型中，神经活动被假定为来自大脑用于概率计算的概率分布的样本。然而，对于神经动力学的机械模型如何能够从任意分布中进行采样，目前仍缺乏全面的理解。我们利用泛函分析和随机微分方程的工具，探索了$\textit{循环}$神经回路从复杂分布中采样所需的最小架构要求。我们首先考虑了由神经元网络组成的传统采样模型，其中神经元的输出直接代表样本（纯采样器网络）。我们认为，传统模型中的突触电流和发放率动力学从复杂概率分布中采样的能力有限。我们证明，具有独立输出单元组的循环神经回路中的发放率动力学可以从任意概率分布中采样。我们将此类回路称为水库采样器网络（RSNs）。我们提出了一种基于去噪分数匹配的高效训练程序，以找到使RSN实现朗之万采样的循环权重和输出权重。我们通过实验证明了我们的模型能够使用所提出的神经动力学从几种复杂数据分布中采样，并讨论了其在开发下一代基于采样的脑模型中的适用性。