Artificial Kuramoto oscillatory neurons were recently introduced as an alternative to threshold units. Empirical evidence suggests that oscillatory units outperform threshold units in several tasks including unsupervised object discovery and certain reasoning problems. The proposed coupling mechanism for these oscillatory neurons is heterogeneous, combining a generalized Kuramoto equation with standard coupling methods used for threshold units. In this research note, we present a theoretical framework that clearly distinguishes oscillatory neurons from threshold units and establishes a coupling mechanism between them. We argue that, from a biological standpoint, oscillatory and threshold units realise distinct aspects of neural coding: roughly, threshold units model intensity of neuron firing, while oscillatory units facilitate information exchange by frequency modulation. To derive interaction between these two types of units, we constrain their dynamics by focusing on dynamical systems that admit Lyapunov functions. For threshold units, this leads to Hopfield associative memory model, and for oscillatory units it yields a specific form of generalized Kuramoto model. The resulting dynamical systems can be naturally coupled to form a Hopfield-Kuramoto associative memory model, which also admits a Lyapunov function. Various forms of coupling are possible. Notably, oscillatory neurons can be employed to implement a low-rank correction to the weight matrix of a Hopfield network. This correction can be viewed either as a form of Hebbian learning or as a popular LoRA method used for fine-tuning of large language models. We demonstrate the practical realization of this particular coupling through illustrative toy experiments.

翻译：人工Kuramoto振荡神经元最近被提出作为阈值单元的一种替代方案。经验证据表明，振荡单元在包括无监督物体发现和某些推理问题在内的多项任务中表现优于阈值单元。针对这些振荡神经元所提出的耦合机制具有异质性，它将广义Kuramoto方程与阈值单元常用的标准耦合方法相结合。在本研究报告中，我们提出了一个理论框架，该框架清晰地区分了振荡神经元与阈值单元，并建立了二者之间的耦合机制。我们认为，从生物学角度来看，振荡单元与阈值单元实现了神经编码的不同方面：粗略而言，阈值单元模拟神经元放电的强度，而振荡单元则通过频率调制促进信息交换。为了推导这两类单元之间的相互作用，我们通过聚焦于允许李雅普诺夫函数的动力系统来约束它们的动力学行为。对于阈值单元，这导出了Hopfield联想记忆模型；对于振荡单元，则得到了一种特定形式的广义Kuramoto模型。所得的动力系统可以自然地耦合形成Hopfield-Kuramoto联想记忆模型，该模型同样允许存在李雅普诺夫函数。多种耦合形式是可能的。值得注意的是，振荡神经元可用于实现对Hopfield网络权重矩阵的低秩修正。这种修正既可视为赫布学习的一种形式，也可视为用于大语言模型微调的流行LoRA方法。我们通过示例性的玩具实验展示了这种特定耦合的实际实现。