The drive to develop artificial neural networks that efficiently utilize resources has generated significant interest in bio-inspired Spiking Neural Networks (SNNs). These networks are particularly attractive due to their potential in applications requiring low power and memory. This potential is further enhanced by the ability to perform online local learning, enabling them to adapt to dynamic environments. This requires the model to be adaptive in a self-supervised manner. While self-supervised learning has seen great success in many deep learning domains, its application for online local learning in multi-layer SNNs remains underexplored. In this paper, we introduce the "EchoSpike Predictive Plasticity" (ESPP) learning rule, a pioneering online local learning rule designed to leverage hierarchical temporal dynamics in SNNs through predictive and contrastive coding. We validate the effectiveness of this approach using benchmark datasets, demonstrating that it performs on par with current state-of-the-art supervised learning rules. The temporal and spatial locality of ESPP makes it particularly well-suited for low-cost neuromorphic processors, representing a significant advancement in developing biologically plausible self-supervised learning models for neuromorphic computing at the edge.
翻译:开发高效利用资源的人工神经网络的需求,引发了人们对仿生脉冲神经网络(SNNs)的浓厚兴趣。这些网络因其在需要低功耗和低内存应用中的潜力而极具吸引力。在线局部学习的能力进一步增强了这种潜力,使其能够适应动态环境。这要求模型能以自监督的方式进行自适应。尽管自监督学习在许多深度学习领域取得了巨大成功,但其在多层SNNs中实现在线局部学习的应用仍未得到充分探索。本文提出了“回声尖峰预测可塑性”(ESPP)学习规则,这是一种开创性的在线局部学习规则,旨在通过预测性和对比性编码,利用SNNs中的层次化时间动态特性。我们使用基准数据集验证了该方法的有效性,证明其性能与当前最先进的监督学习规则相当。ESPP在时间和空间上的局部性使其特别适合低成本神经形态处理器,这代表了在边缘神经形态计算领域开发生物学合理的自监督学习模型方面的一项重大进展。