We investigate the computational limits of the memory retrieval dynamics of modern Hopfield models from the fine-grained complexity analysis. Our key contribution is the characterization of a phase transition behavior in the efficiency of all possible modern Hopfield models based on the norm of patterns. Specifically, we establish an upper bound criterion for the norm of input query patterns and memory patterns. Only below this criterion, sub-quadratic (efficient) variants of the modern Hopfield model exist, assuming the Strong Exponential Time Hypothesis (SETH). To showcase our theory, we provide a formal example of efficient constructions of modern Hopfield models using low-rank approximation when the efficient criterion holds. This includes a derivation of a lower bound on the computational time, scaling linearly with $\Max\{$# of stored memory patterns, length of input query sequence$\}$. In addition, we prove its memory retrieval error bound and exponential memory capacity.

翻译：本文从细粒度复杂度角度研究了现代Hopfield模型记忆检索动态的计算极限。我们的核心贡献在于揭示了所有现代Hopfield模型基于模式范数的效率相变特征。具体而言，我们建立了输入查询模式和记忆模式范数的上限判据。在强指数时间假设（SETH）下，仅当低于该判据时，才存在次二次（高效）的现代Hopfield模型变体。为验证理论，我们展示了当高效判据成立时，利用低秩近似构建高效现代Hopfield模型的正式示例，包括推导出与存储记忆模式数量及输入查询序列长度最大值呈线性比例的计算时间下界。此外，我们还证明了该模型的记忆检索误差界以及指数级记忆容量。