We propose a two-stage memory retrieval dynamics for modern Hopfield models, termed $\mathtt{U\text{-}Hop}$, with enhanced memory capacity. Our key contribution is a learnable feature map $\Phi$ which transforms the Hopfield energy function into kernel space. This transformation ensures convergence between the local minima of energy and the fixed points of retrieval dynamics within the kernel space. Consequently, the kernel norm induced by $\Phi$ serves as a novel similarity measure. It utilizes the stored memory patterns as learning data to enhance memory capacity across all modern Hopfield models. Specifically, we accomplish this by constructing a separation loss $\mathcal{L}_\Phi$ that separates the local minima of kernelized energy by separating stored memory patterns in kernel space. Methodologically, $\mathtt{U\text{-}Hop}$ memory retrieval process consists of: (Stage I) minimizing separation loss for a more uniform memory (local minimum) distribution, followed by (Stage II) standard Hopfield energy minimization for memory retrieval. This results in a significant reduction of possible metastable states in the Hopfield energy function, thus enhancing memory capacity by preventing memory confusion. Empirically, with real-world datasets, we demonstrate that $\mathtt{U\text{-}Hop}$ outperforms all existing modern Hopfield models and state-of-the-art similarity measures, achieving substantial improvements in both associative memory retrieval and deep learning tasks. Code is available at https://github.com/MAGICS-LAB/UHop ; future updates are on arXiv:2404.03827

翻译：我们提出了一种用于现代Hopfield模型的两阶段记忆检索动力学方法，称为$\mathtt{U\text{-}Hop}$，该方法显著提升了记忆容量。核心贡献在于引入可学习特征映射$\Phi$，将Hopfield能量函数转换至核空间。该变换确保了能量局部极小值与核空间内检索动力学不动点的一致性。由此，由$\Phi$诱导的核范数成为一种新型相似性度量，利用存储的记忆模式作为学习数据，全面增强所有现代Hopfield模型的记忆容量。具体实现中，我们构建分离损失函数$\mathcal{L}_\Phi$，通过在核空间中分离存储的记忆模式，实现核化能量局部极小值的分离。方法学上，$\mathtt{U\text{-}Hop}$的记忆检索过程包含两个阶段：第一阶段通过最小化分离损失获得更均匀的记忆（局部极小值）分布；第二阶段执行标准Hopfield能量最小化以完成记忆检索。该机制显著减少了Hopfield能量函数中可能存在的亚稳态数量，通过避免记忆混淆提升容量。实验表明，基于真实数据集，$\mathtt{U\text{-}Hop}$在所有现有现代Hopfield模型及最先进相似性度量中表现最优，在联想记忆检索与深度学习任务中均取得实质性改进。代码见https://github.com/MAGICS-LAB/UHop ；后续更新可参考arXiv:2404.03827。