The majority of research in both training Artificial Neural Networks (ANNs) and modeling learning in biological brains focuses on synaptic plasticity, where learning equates to changing the strength of existing connections. However, in biological brains, structural plasticity - where new connections are created and others removed - is also vital, not only for effective learning but also for recovery from damage and optimal resource usage. Inspired by structural plasticity, pruning is often used in machine learning to remove weak connections from trained models to reduce the computational requirements of inference. However, the machine learning frameworks typically used for backpropagation-based training of both ANNs and Spiking Neural Networks (SNNs) are optimized for dense connectivity, meaning that pruning does not help reduce the training costs of ever-larger models. The GeNN simulator already supports efficient GPU-accelerated simulation of sparse SNNs for computational neuroscience and machine learning. Here, we present a new flexible framework for implementing GPU-accelerated structural plasticity rules and demonstrate this first using the e-prop supervised learning rule and DEEP R to train efficient, sparse SNN classifiers and then, in an unsupervised learning context, to learn topographic maps. Compared to baseline dense models, our sparse classifiers reduce training time by up to 10x while the DEEP R rewiring enables them to perform as well as the original models. We demonstrate topographic map formation in faster-than-realtime simulations, provide insights into the connectivity evolution, and measure simulation speed versus network size. The proposed framework will enable further research into achieving and maintaining sparsity in network structure and neural communication, as well as exploring the computational benefits of sparsity in a range of neuromorphic applications.

翻译：在人工神经网络（ANN）的训练以及生物大脑学习建模的研究中，绝大多数工作聚焦于突触可塑性，即学习等同于改变现有连接的强度。然而，在生物大脑中，结构可塑性——即新连接的创建与旧连接的移除——同样至关重要，这不仅关乎有效学习，也涉及损伤恢复与资源的最优利用。受结构可塑性启发，机器学习中常采用剪枝技术，从已训练模型中移除弱连接以降低推理的计算需求。然而，通常用于基于反向传播训练ANN和脉冲神经网络（SNN）的机器学习框架是针对密集连接优化的，这意味着剪枝无助于减少日益增大的模型的训练成本。GeNN模拟器已支持为计算神经科学和机器学习高效地进行GPU加速的稀疏SNN模拟。本文提出了一种新的灵活框架，用于实现GPU加速的结构可塑性规则，并首先使用e-prop监督学习规则和DEEP R来训练高效、稀疏的SNN分类器，随后在无监督学习场景中学习拓扑映射，以此进行演示。与基线密集模型相比，我们的稀疏分类器将训练时间减少了高达10倍，同时DEEP R重布线使其性能与原模型相当。我们展示了在快于实时模拟中的拓扑映射形成，深入分析了连接性的演化过程，并测量了模拟速度与网络规模的关系。所提出的框架将推动在网络结构与神经通信中实现和维持稀疏性，以及探索稀疏性在一系列神经形态应用中的计算优势方面的进一步研究。