The increasing computational demands of modern AI systems have exposed fundamental limitations of digital hardware, driving interest in alternative paradigms for efficient large-scale inference. Dense Associative Memory (DenseAM) is a family of models that offers a flexible framework for representing many contemporary neural architectures, such as transformers and diffusion models, by casting them as dynamical systems evolving on an energy landscape. In this work, we propose a general method for building analog accelerators for DenseAMs and implementing them using electronic RC circuits, crossbar arrays, and amplifiers. We find that our analog DenseAM hardware performs inference in constant time independent of model size. This result highlights an asymptotic advantage of analog DenseAMs over digital numerical solvers that scale at least linearly with the model size. We consider three settings of progressively increasing complexity: XOR, the Hamming (7,4) code, and a simple language model defined on binary variables. We propose analog implementations of these three models and analyze the scaling of inference time, energy consumption, and hardware. Finally, we estimate lower bounds on the achievable time constants imposed by amplifier specifications, suggesting that even conservative existing analog technology can enable inference times on the order of tens to hundreds of nanoseconds. By harnessing the intrinsic parallelism and continuous-time operation of analog circuits, our DenseAM-based accelerator design offers a new avenue for fast and scalable AI hardware.

翻译：现代人工智能系统日益增长的计算需求暴露了数字硬件的根本局限性，推动了对高效大规模推理替代范式的兴趣。密集关联存储器（DenseAM）是一类模型，通过将诸如Transformer和扩散模型等当代神经架构视为在能量景观上演化的动力系统，为表示这些架构提供了灵活的框架。在本研究中，我们提出了一种构建DenseAM模拟加速器的通用方法，并利用电子RC电路、交叉阵列和放大器实现它们。我们发现，我们的模拟DenseAM硬件执行推理的时间是恒定的，与模型大小无关。这一结果凸显了模拟DenseAM相对于数字数值求解器的渐近优势，后者至少随模型大小线性扩展。我们考虑了复杂度逐步增加的三种场景：XOR、汉明（7,4）码和基于二元变量的简单语言模型。我们提出了这三种模型的模拟实现，并分析了推理时间、能耗和硬件的扩展性。最后，我们估算了由放大器规格所施加的可实现时间常数的下界，表明即使是现有保守的模拟技术也能实现数十到数百纳秒量级的推理时间。通过利用模拟电路固有的并行性和连续时间操作，我们基于DenseAM的加速器设计为快速可扩展的人工智能硬件开辟了新途径。