Neuromorphic computing with spiking neural networks is promising for energy-efficient artificial intelligence (AI) applications. However, different from humans who continually learn different tasks in a lifetime, neural network models suffer from catastrophic forgetting. How could neuronal operations solve this problem is an important question for AI and neuroscience. Many previous studies draw inspiration from observed neuroscience phenomena and propose episodic replay or synaptic metaplasticity, but they are not guaranteed to explicitly preserve knowledge for neuron populations. Other works focus on machine learning methods with more mathematical grounding, e.g., orthogonal projection on high dimensional spaces, but there is no neural correspondence for neuromorphic computing. In this work, we develop a new method with neuronal operations based on lateral connections and Hebbian learning, which can protect knowledge by projecting activity traces of neurons into an orthogonal subspace so that synaptic weight update will not interfere with old tasks. We show that Hebbian and anti-Hebbian learning on recurrent lateral connections can effectively extract the principal subspace of neural activities and enable orthogonal projection. This provides new insights into how neural circuits and Hebbian learning can help continual learning, and also how the concept of orthogonal projection can be realized in neuronal systems. Our method is also flexible to utilize arbitrary training methods based on presynaptic activities/traces. Experiments show that our method consistently solves forgetting for spiking neural networks with nearly zero forgetting under various supervised training methods with different error propagation approaches, and outperforms previous approaches under various settings. Our method can pave a solid path for building continual neuromorphic computing systems.

翻译：神经形态计算结合脉冲神经网络在能效型人工智能应用中具有广阔前景。然而，与人类能够终身持续学习不同任务的能力不同，神经网络模型饱受灾难性遗忘的困扰。神经元操作如何解决这一问题，是人工智能和神经科学领域的重要课题。许多先前研究从观察到的神经科学现象中汲取灵感，提出情景回放或突触可塑性机制，但这些方法不能明确保证神经元群体知识的持久保存。另一类研究侧重于更具数学基础的机器学习方法，例如高维空间的正交投影，但这些方法缺乏适用于神经形态计算的神经对应机制。在本研究中，我们开发了一种基于侧向连接和赫布学习的神经元操作新方法，通过将神经元活动轨迹投影到正交子空间来保护知识，从而确保突触权重更新不会干扰旧任务。我们证明，递归侧向连接上的赫布学习和反赫布学习能够有效提取神经活动的主子空间，并实现正交投影。这为理解神经回路和赫布学习如何促进持续学习，以及正交投影概念如何在神经元系统中实现提供了新见解。我们的方法还能灵活利用基于突触前活动轨迹的任意训练方法。实验表明，在采用不同误差传播途径的多种监督训练方法下，该方法能始终如一地解决脉冲神经网络的遗忘问题，实现近乎零遗忘，并在多种设置下优于先前方法。该方法可为构建持续神经形态计算系统奠定坚实基础。