Hopfield networks are widely used in neuroscience as simplified theoretical models of biological associative memory. The original Hopfield networks store memories by encoding patterns of binary associations, which result in a synaptic learning mechanism known as Hebbian learning rule. Modern Hopfield networks can achieve exponential capacity scaling by using highly non-linear energy functions. However, the energy function of these newer models cannot be straightforwardly compressed into binary synaptic couplings and it does not directly provide new synaptic learning rules. In this work we show that generative diffusion models can be interpreted as energy-based models and that, when trained on discrete patterns, their energy function is equivalent to that of modern Hopfield networks. This equivalence allows us to interpret the supervised training of diffusion models as a synaptic learning process that encodes the associative dynamics of a modern Hopfield network in the weight structure of a deep neural network. Accordingly, in our experiments we show that the storage capacity of a continuous modern Hopfield network is identical to the capacity of a diffusion model. Our results establish a strong link between generative modeling and the theoretical neuroscience of memory, which provide a powerful computational foundation for the reconstructive theory of memory, where creative generation and memory recall can be seen as parts of a unified continuum.

翻译：霍普菲尔德网络在神经科学中被广泛用作生物联想记忆的简化理论模型。原始霍普菲尔德网络通过编码二进制关联模式来存储记忆，由此产生了一种称为赫布学习规则的突触学习机制。现代霍普菲尔德网络通过使用高度非线性的能量函数，能够实现指数级容量扩展。然而，这些新模型的能量函数无法直接压缩为二进制突触耦合，也无法直接提供新的突触学习规则。在本工作中，我们证明生成扩散模型可被解释为基于能量的模型，并且当在离散模式上训练时，其能量函数等价于现代霍普菲尔德网络的能量函数。这一等价性使我们能够将扩散模型的监督训练解释为一个突触学习过程，该过程将现代霍普菲尔德网络的联想动力学编码到深度神经网络的权重结构中。相应地，我们的实验表明连续型现代霍普菲尔德网络的存储容量与扩散模型的容量相同。我们的结果在生成建模与记忆的理论神经科学之间建立了强关联，这为记忆的重构理论提供了强大的计算基础，在该理论中，创造性生成与记忆回忆可被视为统一连续体中的组成部分。