We propose a two-stage memory retrieval dynamics for modern Hopfield models, termed $\mathtt{U\text{-}Hop}$, with enhanced memory capacity. Our key contribution is a learnable feature map $\Phi$ which transforms the Hopfield energy function into kernel space. This transformation ensures convergence between the local minima of energy and the fixed points of retrieval dynamics within the kernel space. Consequently, the kernel norm induced by $\Phi$ serves as a novel similarity measure. It utilizes the stored memory patterns as learning data to enhance memory capacity across all modern Hopfield models. Specifically, we accomplish this by constructing a separation loss $\mathcal{L}_\Phi$ that separates the local minima of kernelized energy by separating stored memory patterns in kernel space. Methodologically, $\mathtt{U\text{-}Hop}$ memory retrieval process consists of: (Stage I) minimizing separation loss for a more uniform memory (local minimum) distribution, followed by (Stage II) standard Hopfield energy minimization for memory retrieval. This results in a significant reduction of possible metastable states in the Hopfield energy function, thus enhancing memory capacity by preventing memory confusion. Empirically, with real-world datasets, we demonstrate that $\mathtt{U\text{-}Hop}$ outperforms all existing modern Hopfield models and state-of-the-art similarity measures, achieving substantial improvements in both associative memory retrieval and deep learning tasks. Code is available at https://github.com/MAGICS-LAB/UHop ; future updates are on arXiv:2404.03827

翻译：我们为现代Hopfield模型提出了一种具有增强记忆容量的两阶段记忆检索动力学方法，称为$\mathtt{U\text{-}Hop}$。我们的核心贡献是一个可学习的特征映射$\Phi$，它将Hopfield能量函数转换到核空间。该变换确保了核空间中能量局部极小值与检索动力学不动点之间的收敛性。因此，由$\Phi$诱导的核范数成为一种新型相似性度量。该方法利用存储的记忆模式作为学习数据，以提升所有现代Hopfield模型的记忆容量。具体而言，我们通过构建分离损失$\mathcal{L}_\Phi$来实现这一目标，该损失通过在核空间中分离存储的记忆模式来分离核化能量的局部极小值。从方法论上，$\mathtt{U\text{-}Hop}$记忆检索过程包含两个阶段：（阶段Ⅰ）最小化分离损失以获得更均匀的记忆（局部极小值）分布，随后（阶段Ⅱ）执行标准Hopfield能量最小化以实现记忆检索。这显著减少了Hopfield能量函数中可能存在的亚稳态，从而通过避免记忆混淆来提升记忆容量。基于真实世界数据集的实验表明，$\mathtt{U\text{-}Hop}$在所有现有现代Hopfield模型和最先进的相似性度量方法中均表现出优越性，在联想记忆检索和深度学习任务中均取得显著提升。代码发布于https://github.com/MAGICS-LAB/UHop；后续更新详见arXiv:2404.03827。