We investigate the computational limits of the memory retrieval dynamics of modern Hopfield models from the fine-grained complexity analysis. Our key contribution is the characterization of a phase transition behavior in the efficiency of all possible modern Hopfield models based on the norm of patterns. Specifically, we establish an upper bound criterion for the norm of input query patterns and memory patterns. Only below this criterion, sub-quadratic (efficient) variants of the modern Hopfield model exist, assuming the Strong Exponential Time Hypothesis (SETH). To showcase our theory, we provide a formal example of efficient constructions of modern Hopfield models using low-rank approximation when the efficient criterion holds. This includes a derivation of a lower bound on the computational time, scaling linearly with $\Max\{$# of stored memory patterns, length of input query sequence$\}$. In addition, we prove its memory retrieval error bound and exponential memory capacity.

翻译：我们从细粒度复杂性分析的角度，研究了现代Hopfield模型记忆检索动态过程的计算极限。我们的核心贡献在于，基于模式的范数，刻画了所有可能现代Hopfield模型效率中的一个相变行为。具体地，我们为输入查询模式和记忆模式建立了范数的上界准则。只有当这一准则满足时，在强指数时间假设（SETH）下，次二次（高效）的现代Hopfield模型变体才可能存在。为展示我们的理论，我们提供了一个形式化示例，说明当高效准则成立时，如何利用低秩近似构建高效的现代Hopfield模型。这包括推导一个下界计算时间，其随$\Max\{$存储记忆模式数量，输入查询序列长度$\}$线性增长。此外，我们证明了其记忆检索误差界以及指数级记忆容量。