We investigate the computational limits of the memory retrieval dynamics of modern Hopfield models from the fine-grained complexity analysis. Our key contribution is the characterization of a phase transition behavior in the efficiency of all possible modern Hopfield models based on the norm of patterns. Specifically, we establish an upper bound criterion for the norm of input query patterns and memory patterns. Only below this criterion, sub-quadratic (efficient) variants of the modern Hopfield model exist, assuming the Strong Exponential Time Hypothesis (SETH). To showcase our theory, we provide a formal example of efficient constructions of modern Hopfield models using low-rank approximation when the efficient criterion holds. This includes a derivation of a lower bound on the computational time, scaling linearly with $\max\{$# of stored memory patterns, length of input query sequence$\}$. In addition, we prove its memory retrieval error bound and exponential memory capacity.

翻译：我们从细粒度复杂度分析的视角研究了现代Hopfield模型记忆检索动态的计算极限。我们的核心贡献在于基于模式范数刻画了所有可能的现代Hopfield模型在计算效率上的相变行为。具体而言，我们建立了输入查询模式与记忆模式范数的上界判据。在强指数时间假设（SETH）条件下，仅当满足该判据时，才存在具有亚二次时间复杂度（高效）的现代Hopfield模型变体。为验证理论，我们给出了一个形式化示例：当高效性判据成立时，通过低秩近似方法高效构建现代Hopfield模型。这包括推导出计算时间的下界，该下界随$\max\{$存储的记忆模式数量, 输入查询序列长度$\}$线性增长。此外，我们证明了其记忆检索误差界与指数级记忆容量。