Modern Hopfield Networks (MHNs) have emerged as powerful components in deep learning, serving as effective replacements for pooling layers, LSTMs, and attention mechanisms. While recent advancements have significantly improved their storage capacity and retrieval efficiency, their fundamental theoretical boundaries remain underexplored. In this paper, we rigorously characterize the expressive power of MHNs through the lens of circuit complexity theory. We prove that $\mathrm{poly}(n)$-precision MHNs with constant depth and linear hidden dimension fall within the $\mathsf{DLOGTIME}$-uniform $\mathsf{TC}^0$ complexity class. Consequently, assuming $\mathsf{TC}^0 \neq \mathsf{NC}^1$, we demonstrate that these architectures are incapable of solving $\mathsf{NC}^1$-hard problems, such as undirected graph connectivity and tree isomorphism. We further extend these impossibility results to Kernelized Hopfield Networks. However, we show that these limitations are not absolute: we prove that equipping MHNs with a Chain-of-Thought (CoT) mechanism enables them to transcend the $\mathsf{TC}^0$ barrier, allowing them to solve inherently serial problems like the word problem for the permutation group $S_5$. Collectively, our results delineate a fine-grained boundary between the capabilities of standard MHNs and those augmented with reasoning steps.

翻译：现代Hopfield网络（MHNs）已成为深度学习中的强大组件，可有效替代池化层、LSTM和注意力机制。尽管近期进展显著提升了其存储容量与检索效率，但其基本理论边界仍未得到充分探索。本文通过电路复杂性理论的视角，严格刻画了MHNs的表达能力。我们证明具有常数深度与线性隐藏维度的$\mathrm{poly}(n)$精度MHNs属于$\mathsf{DLOGTIME}$-均匀$\mathsf{TC}^0$复杂性类。因此，在假设$\mathsf{TC}^0 \neq \mathsf{NC}^1$的前提下，我们证明此类架构无法解决$\mathsf{NC}^1$-难问题，例如无向图连通性与树同构判定。我们进一步将这类不可能性结果推广至核化Hopfield网络。然而，这些局限性并非绝对：我们证明为MHNs配备思维链（CoT）机制可使其突破$\mathsf{TC}^0$屏障，从而能够解决本质串行问题，例如置换群$S_5$的字问题。综合而言，我们的研究结果清晰划定了标准MHNs与增强推理步骤的MHNs之间的精细能力边界。