The backpropagation algorithm has promoted the rapid development of deep learning, but it relies on a large amount of labeled data and still has a large gap with how humans learn. The human brain can quickly learn various conceptual knowledge in a self-organized and unsupervised manner, accomplished through coordinating various learning rules and structures in the human brain. Spike-timing-dependent plasticity (STDP) is a general learning rule in the brain, but spiking neural networks (SNNs) trained with STDP alone is inefficient and perform poorly. In this paper, taking inspiration from short-term synaptic plasticity, we design an adaptive synaptic filter and introduce the adaptive spiking threshold as the neuron plasticity to enrich the representation ability of SNNs. We also introduce an adaptive lateral inhibitory connection to adjust the spikes balance dynamically to help the network learn richer features. To speed up and stabilize the training of unsupervised spiking neural networks, we design a samples temporal batch STDP (STB-STDP), which updates weights based on multiple samples and moments. By integrating the above three adaptive mechanisms and STB-STDP, our model greatly accelerates the training of unsupervised spiking neural networks and improves the performance of unsupervised SNNs on complex tasks. Our model achieves the current state-of-the-art performance of unsupervised STDP-based SNNs in the MNIST and FashionMNIST datasets. Further, we tested on the more complex CIFAR10 dataset, and the results fully illustrate the superiority of our algorithm. Our model is also the first work to apply unsupervised STDP-based SNNs to CIFAR10. At the same time, in the small-sample learning scenario, it will far exceed the supervised ANN using the same structure.
翻译:反向传播算法推动了深度学习的快速发展,但其依赖于大量标注数据,且与人类学习方式仍存在巨大差距。人脑能够以自组织无监督的方式快速学习各种概念性知识,这是通过协调人脑中多种学习规则与结构实现的。脉冲时序依赖可塑性(STDP)是大脑中普遍存在的学习规则,但仅使用STDP训练的脉冲神经网络(SNN)效率低下且性能较差。本文受短期突触可塑性启发,设计了自适应突触滤波器,并引入自适应脉冲阈值作为神经元可塑性,以增强SNN的表征能力。同时引入自适应侧向抑制连接动态调节脉冲平衡,帮助网络学习更丰富的特征。为加速并稳定无监督脉冲神经网络的训练,我们设计了样本时序批处理STDP(STB-STDP),该机制基于多个样本和时刻更新权重。通过整合上述三种自适应机制与STB-STDP,我们的模型极大加速了无监督脉冲神经网络的训练,并提升了其在复杂任务上的性能。在MNIST和FashionMNIST数据集上,我们的模型达到了当前基于无监督STDP的SNN的最优性能。此外,我们在更复杂的CIFAR10数据集上进行了测试,结果充分证明了算法的优越性。我们的模型也是首个将基于无监督STDP的SNN应用于CIFAR10的工作。同时,在小样本学习场景中,其性能远超使用相同结构的监督人工神经网络(ANN)。