In sampling-based Bayesian models of brain function, neural activities are assumed to be samples from probability distributions that the brain uses for probabilistic computation. However, a comprehensive understanding of how mechanistic models of neural dynamics can sample from arbitrary distributions is still lacking. We use tools from functional analysis and stochastic differential equations to explore the minimum architectural requirements for $\textit{recurrent}$ neural circuits to sample from complex distributions. We first consider the traditional sampling model consisting of a network of neurons whose outputs directly represent the samples (sampler-only network). We argue that synaptic current and firing-rate dynamics in the traditional model have limited capacity to sample from a complex probability distribution. We show that the firing rate dynamics of a recurrent neural circuit with a separate set of output units can sample from an arbitrary probability distribution. We call such circuits reservoir-sampler networks (RSNs). We propose an efficient training procedure based on denoising score matching that finds recurrent and output weights such that the RSN implements Langevin sampling. We empirically demonstrate our model's ability to sample from several complex data distributions using the proposed neural dynamics and discuss its applicability to developing the next generation of sampling-based brain models.

翻译：在基于采样的脑功能贝叶斯模型中，神经活动被视为大脑用于概率计算的概率分布样本。然而，目前仍缺乏对神经动力学机制模型如何从任意分布中采样的全面理解。我们利用泛函分析和随机微分方程工具，探究递归神经回路从复杂分布中采样的最小架构需求。首先考虑由神经元输出直接代表样本的传统采样模型（纯采样网络）。我们论证传统模型中突触电流和发放率动力学在从复杂概率分布采样时存在能力局限。研究表明，具有独立输出单元的递归神经回路发放率动力学能够从任意概率分布中采样。我们将此类回路称为储层采样器网络（RSN）。我们提出一种基于去噪分数匹配的高效训练方法，通过确定递归权重与输出权重使RSN实现朗之万采样。通过实证验证，该模型能利用所提出的神经动力学从多种复杂数据分布中采样，并讨论了其对于开发下一代基于采样的脑模型的适用性。