Stochastic diffusion processes are pervasive in nature, from the seemingly erratic Brownian motion to the complex interactions of synaptically-coupled spiking neurons. Recently, drawing inspiration from Langevin dynamics, neuromorphic diffusion models were proposed and have become one of the major breakthroughs in the field of generative artificial intelligence. Unlike discriminative models that have been well developed to tackle classification or regression tasks, diffusion models as well as other generative models such as ChatGPT aim at creating content based upon contexts learned. However, the more complex algorithms of these models result in high computational costs using today's technologies, creating a bottleneck in their efficiency, and impeding further development. Here, we develop a spintronic voltage-controlled magnetoelectric memory hardware for the neuromorphic diffusion process. The in-memory computing capability of our spintronic devices goes beyond current Von Neumann architecture, where memory and computing units are separated. Together with the non-volatility of magnetic memory, we can achieve high-speed and low-cost computing, which is desirable for the increasing scale of generative models in the current era. We experimentally demonstrate that the hardware-based true random diffusion process can be implemented for image generation and achieve comparable image quality to software-based training as measured by the Frechet inception distance (FID) score, achieving ~10^3 better energy-per-bit-per-area over traditional hardware.

翻译：随机扩散过程在自然界中无处不在，从看似无规则的布朗运动到突触耦合的脉冲神经元的复杂相互作用。近期，受朗之万动力学启发，神经形态扩散模型被提出并已成为生成式人工智能领域的主要突破之一。与已充分发展用于处理分类或回归任务的判别式模型不同，扩散模型以及诸如ChatGPT等其他生成式模型，旨在基于学习到的上下文内容进行创作。然而，这些模型更复杂的算法导致使用现有技术时计算成本高昂，形成了效率瓶颈，阻碍了进一步发展。在此，我们为神经形态扩散过程开发了一种自旋电子电压控制磁电存储器硬件。我们自旋电子器件的存内计算能力超越了当前内存与计算单元分离的冯·诺依曼架构。结合磁性存储器的非易失性，我们可以实现高速、低成本的计算，这对于当前时代不断扩大的生成式模型规模而言是至关重要的。我们通过实验证明，基于硬件的真实随机扩散过程可用于图像生成，并且以弗雷歇起始距离（FID）分数衡量，其图像质量可与基于软件的训练相媲美，同时实现了比传统硬件每比特每面积能量效率提升约10^3倍。