Hierarchical Associative Memory models have recently been proposed as a versatile extension of continuous Hopfield networks. In order to facilitate future research on such models, especially at scale, we focus on increasing their simulation efficiency on digital hardware. In particular, we propose two strategies to speed up memory retrieval in these models, which corresponds to their use at inference, but is equally important during training. First, we show how they can be cast as Deep Equilibrium Models, which allows using faster and more stable solvers. Second, inspired by earlier work, we show that alternating optimization of the even and odd layers accelerates memory retrieval by a factor close to two. Combined, these two techniques allow for a much faster energy minimization, as shown in our proof-of-concept experimental results. The code is available at https://github.com/cgoemaere/hamdeq

翻译：层次联想记忆模型最近被提出作为连续Hopfield网络的一种通用扩展。为促进此类模型（尤其是大规模场景）的未来研究，我们聚焦于提升其在数字硬件上的仿真效率。具体而言，我们提出两种策略来加速这些模型中的记忆检索过程——该过程既对应于推理时的使用，在训练中也同样重要。首先，我们展示了如何将其转化为深度平衡模型，从而能够使用更快且更稳定的求解器。其次，受早期工作启发，我们证明交替优化偶数和奇数层可将记忆检索速度提升近两倍。结合这两种技术可实现更快速的能量最小化，概念验证实验结果已证实这一点。代码托管于 https://github.com/cgoemaere/hamdeq