The Hopfield model is a paradigmatic model of neural networks that has been analyzed for many decades in the statistical physics, neuroscience, and machine learning communities. Inspired by the manifold hypothesis in machine learning, we propose and investigate a generalization of the standard setting that we name Random-Features Hopfield Model. Here $P$ binary patterns of length $N$ are generated by applying to Gaussian vectors sampled in a latent space of dimension $D$ a random projection followed by a non-linearity. Using the replica method from statistical physics, we derive the phase diagram of the model in the limit $P,N,D\to\infty$ with fixed ratios $\alpha=P/N$ and $\alpha_D=D/N$. Besides the usual retrieval phase, where the patterns can be dynamically recovered from some initial corruption, we uncover a new phase where the features characterizing the projection can be recovered instead. We call this phenomena the learning phase transition, as the features are not explicitly given to the model but rather are inferred from the patterns in an unsupervised fashion.

翻译：Hopfield模型是神经网络的一个经典模型，已在统计物理、神经科学和机器学习领域被分析了数十年。受机器学习中流形假设的启发，我们提出并研究了一种标准设置的推广，命名为随机特征Hopfield模型。在该模型中，$P$个长度为$N$的二元模式通过以下方式生成：对在维度为$D$的潜空间中采样的高斯向量应用随机投影，随后进行非线性变换。利用统计物理中的复制方法，我们在极限条件$P,N,D\to\infty$且固定比值$\alpha=P/N$和$\alpha_D=D/N$下推导出该模型的相图。除了常见的检索相（即模式可以从某些初始扰动中动态恢复）之外，我们发现了一个新相——在该相中，表征投影的特征本身可以被恢复。我们将这种现象称为学习相变，因为特征并非显式地提供给模型，而是以无监督的方式从模式中推断出来。