Developmental plasticity plays a prominent role in shaping the brain's structure during ongoing learning in response to dynamically changing environments. However, the existing network compression methods for deep artificial neural networks (ANNs) and spiking neural networks (SNNs) draw little inspiration from the brain's developmental plasticity mechanisms, thus limiting their ability to learn efficiently, rapidly, and accurately. This paper proposed a developmental plasticity-inspired adaptive pruning (DPAP) method, with inspiration from the adaptive developmental pruning of dendritic spines, synapses, and neurons according to the "use it or lose it, gradually decay" principle. The proposed DPAP model considers multiple biologically realistic mechanisms (such as dendritic spine dynamic plasticity, activity-dependent neural spiking trace, and local synaptic plasticity), with the addition of an adaptive pruning strategy, so that the network structure can be dynamically optimized during learning without any pre-training and retraining. We demonstrated that the proposed DPAP method applied to deep ANNs and SNNs could learn efficient network architectures. Extensive comparative experiments show consistent and remarkable performance and speed boost with the extremely compressed networks on a diverse set of benchmark tasks, especially neuromorphic datasets for SNNs. This work explores how developmental plasticity enables the complex deep networks to gradually evolve into brain-like efficient and compact structures, eventually achieving state-of-the-art (SOTA) performance for biologically realistic SNNs.

翻译：发展可塑性在应对动态变化环境的持续学习过程中，对塑造大脑结构起着重要作用。然而，现有深度人工神经网络（ANN）和脉冲神经网络（SNN）的网络压缩方法极少借鉴大脑的发展可塑性机制，从而限制了其高效、快速且精准学习的能力。本文提出了一种受发展可塑性启发的自适应剪枝（DPAP）方法，该方法借鉴了树突棘、突触和神经元依据"用进废退、逐渐衰退"原则进行的适应性发育剪枝。所提出的DPAP模型融合了多种生物逼真机制（如树突棘动态可塑性、活动依赖的神经脉冲迹以及局部突触可塑性），并辅以自适应剪枝策略，从而使得网络结构能够在学习过程中无需预训练和重训练即可动态优化。我们证明了将所提出的DPAP方法应用于深度ANN和SNN时，能够学习到高效的网络架构。大量对比实验表明，在多种基准任务（尤其是针对SNN的神经形态数据集）上，经过极致压缩的网络持续展现出卓越且稳定的性能提升与速度增长。本研究探索了发展可塑性如何使复杂深度网络逐步演化为类脑的高效紧凑结构，最终使生物逼真的SNN达到最先进（SOTA）性能。