Recent generalizations of the Hopfield model of associative memories are able to store a number $P$ of random patterns that grows exponentially with the number $N$ of neurons, $P=\exp(\alpha N)$. Besides the huge storage capacity, another interesting feature of these networks is their connection to the attention mechanism which is part of the Transformer architectures widely applied in deep learning. In this work, we consider a generic family of pattern ensembles, and thanks to the statistical mechanics analysis of an auxiliary Random Energy Model, we are able to provide exact asymptotic thresholds for the retrieval of a typical pattern, $\alpha_1$, and lower bounds for the maximum of the load $\alpha$ for which all patterns can be retrieved, $\alpha_c$. Additionally, we characterize the size of the basins of attractions. We discuss in detail the cases of Gaussian and spherical patterns, and show that they display rich and qualitatively different phase diagrams.

翻译：近期对联想记忆Hopfield模型的推广能够存储随机模式的数量$P$随神经元数量$N$呈指数增长，即$P=\exp(\alpha N)$。除了巨大的存储容量，这些网络的另一个有趣特性是它们与注意力机制的联系，该机制是深度学习中广泛应用的Transformer架构的组成部分。在本工作中，我们考虑一类通用的模式系综，并借助辅助随机能量模型的统计力学分析，能够给出典型模式检索的精确渐近阈值$\alpha_1$，以及所有模式均可检索时负载$\alpha$最大值的下界$\alpha_c$。此外，我们刻画了吸引域的大小。我们详细讨论了高斯模式和球面模式的情形，并表明它们呈现出丰富且性质迥异的相图。