It has long been known in both neuroscience and AI that ``binding'' between neurons leads to a form of competitive learning where representations are compressed in order to represent more abstract concepts in deeper layers of the network. More recently, it was also hypothesized that dynamic (spatiotemporal) representations play an important role in both neuroscience and AI. Building on these ideas, we introduce Artificial Kuramoto Oscillatory Neurons (AKOrN) as a dynamical alternative to threshold units, which can be combined with arbitrary connectivity designs such as fully connected, convolutional, or attentive mechanisms. Our generalized Kuramoto updates bind neurons together through their synchronization dynamics. We show that this idea provides performance improvements across a wide spectrum of tasks such as unsupervised object discovery, adversarial robustness, calibrated uncertainty quantification, and reasoning. We believe that these empirical results show the importance of rethinking our assumptions at the most basic neuronal level of neural representation, and in particular show the importance of dynamical representations.

翻译：长期以来，神经科学与人工智能领域均已知晓神经元间的"绑定"机制会导致一种竞争性学习形式，即网络深层通过压缩表征来表示更抽象的概念。近期研究进一步提出动态（时空）表征在神经科学与人工智能中均具有重要作用。基于这些理念，我们提出人工Kuramoto振荡神经元（AKOrN）作为阈值单元的动力学替代方案，该单元可与任意连接架构（如全连接、卷积或注意力机制）相结合。我们提出的广义Kuramoto更新规则通过同步动力学实现神经元间的绑定。实验表明，该方法在无监督物体发现、对抗鲁棒性、校准不确定性量化及推理等广泛任务中均能提升性能。我们认为这些实证结果揭示了在神经表征的最基础神经元层面重新审视理论假设的重要性，并特别证明了动态表征的关键作用。