Networks in machine learning offer examples of complex high-dimensional dynamical systems reminiscent of biological systems. Here, we study the learning dynamics of Generalized Hopfield networks, which permit a visualization of internal memories. These networks have been shown to proceed through a 'feature-to-prototype' transition, as the strength of network nonlinearity is increased, wherein the learned, or terminal, states of internal memories transition from mixed to pure states. Focusing on the prototype learning dynamics of the internal memories we observe a strong resemblance to the canalized, or low-dimensional, dynamics of cells as they differentiate within a Waddingtonian landscape. Dynamically, we demonstrate that learning in a Generalized Hopfield Network proceeds through sequential 'splits' in memory space. Furthermore, order of splitting is interpretable and reproducible. The dynamics between the splits are canalized in the Waddington sense -- robust to variations in detailed aspects of the system. In attempting to make the analogy a rigorous equivalence, we study smaller subsystems that exhibit similar properties to the full system. We combine analytical calculations with numerical simulations to study the dynamical emergence of the feature-to-prototype transition, and the behaviour of splits in the landscape, saddles points, visited during learning. We exhibit regimes where saddles appear and disappear through saddle-node bifurcations, qualitatively changing the distribution of learned memories as the strength of the nonlinearity is varied -- allowing us to systematically investigate the mechanisms that underlie the emergence of Waddingtonian dynamics. Memories can thus differentiate in a predictive and controlled way, revealing new bridges between experimental biology, dynamical systems theory, and machine learning.

翻译：机器学习中的网络提供了类似于生物系统的复杂高维动力系统实例。本文研究了广义Hopfield网络的学习动力学，该类网络能够可视化内部记忆。研究表明，随着网络非线性强度增加，这些网络会经历从"特征到原型"的转变，其中内部记忆的学习（或终端）状态从混合态转变为纯态。通过聚焦内部记忆的原型学习动力学，我们观察到其与细胞在Waddington景观中分化时的低维沟道化动力学具有高度相似性。从动力学角度，我们证明广义Hopfield网络中的学习过程通过记忆空间中的连续"分叉"实现，且分裂顺序具有可解释性和可重复性。分裂间的动力学在Waddington意义下呈沟道化特征——对系统细节变化具有鲁棒性。为严格建立这种类比关系，我们研究了与完整系统性质相似的小规模子系统。结合解析计算与数值模拟，我们探究了特征到原型转变的动力学涌现过程，以及学习过程中景观内分裂与鞍点行为。我们展示了当鞍点通过鞍结分岔出现和消失的机制，这些机制随着非线性强度变化会定性改变学习记忆的分布——从而系统揭示了Waddington动力学涌现的底层机理。记忆由此可以预测性、受控地分化，为实验生物学、动力系统理论与机器学习之间搭建了新的桥梁。