Uncovering the mechanisms behind long-term memory is one of the most fascinating open problems in neuroscience and artificial intelligence. Artificial associative memory networks have been used to formalize important aspects of biological memory. Generative diffusion models are a type of generative machine learning techniques that have shown great performance in many tasks. Like associative memory systems, these networks define a dynamical system that converges to a set of target states. In this work we show that generative diffusion models can be interpreted as energy-based models and that, when trained on discrete patterns, their energy function is (asymptotically) identical to that of modern Hopfield networks. This equivalence allows us to interpret the supervised training of diffusion models as a synaptic learning process that encodes the associative dynamics of a modern Hopfield network in the weight structure of a deep neural network. Leveraging this connection, we formulate a generalized framework for understanding the formation of long-term memory, where creative generation and memory recall can be seen as parts of a unified continuum.

翻译：揭示长期记忆背后的机制是神经科学和人工智能领域中最引人入胜的开放性问题之一。人工联想记忆网络已被用于形式化生物学记忆的重要方面。生成扩散模型是一类生成式机器学习技术，在众多任务中展现出卓越性能。与联想记忆系统类似，这些网络定义了一个收敛至一组目标状态的动力学系统。在本研究中，我们证明生成扩散模型可被解释为基于能量的模型，并且当在离散模式上训练时，其能量函数（渐近地）与现代Hopfield网络的能量函数相同。这一等价性使我们能够将扩散模型的监督训练解释为一种突触学习过程——该过程将现代Hopfield网络的联想动力学编码至深度神经网络的权重结构中。基于这一关联，我们构建了一个理解长期记忆形成的广义框架，其中创造性生成与记忆回忆可被视为统一连续谱的两个组成部分。