In both machine learning and in computational neuroscience, plasticity in functional neural networks is frequently expressed as gradient descent on a cost. Often, this imposes symmetry constraints that are difficult to reconcile with local computation, as is required for biological networks or neuromorphic hardware. For example, wake-sleep learning in networks characterized by Boltzmann distributions assumes symmetric connectivity. Similarly, the error backpropagation algorithm is notoriously plagued by the weight transport problem between the representation and the error stream. Existing solutions such as feedback alignment circumvent the problem by deferring to the robustness of these algorithms to weight asymmetry. However, they scale poorly with network size and depth. We introduce spike-based alignment learning (SAL), a complementary learning rule for spiking neural networks, which uses spike timing statistics to extract and correct the asymmetry between effective reciprocal connections. Apart from being spike-based and fully local, our proposed mechanism takes advantage of noise. Based on an interplay between Hebbian and anti-Hebbian plasticity, synapses can thereby recover the true local gradient. This also alleviates discrepancies that arise from neuron and synapse variability -- an omnipresent property of physical neuronal networks. We demonstrate the efficacy of our mechanism using different spiking network models. First, SAL can significantly improve convergence to the target distribution in probabilistic spiking networks versus Hebbian plasticity alone. Second, in neuronal hierarchies based on cortical microcircuits, SAL effectively aligns feedback weights to the forward pathway, thus allowing the backpropagation of correct feedback errors. Third, our approach enables competitive performance in deep networks using only local plasticity for weight transport.

翻译：在机器学习与计算神经科学中，功能性神经网络的塑性常被表述为基于代价函数的梯度下降。然而，这种方法往往施加了与局部计算相矛盾的对称性约束——而局部计算正是生物神经网络或神经形态硬件所要求的。例如，基于玻尔兹曼分布的网络的唤醒-睡眠学习假设了对称连接性；同样，误差反向传播算法长期受到表征流与误差流之间权重传输问题的困扰。现有的解决方案如反馈对齐通过依赖算法对权重不对称性的鲁棒性来规避该问题，但其可扩展性随网络规模与深度增加而显著下降。我们提出基于脉冲的对齐学习（SAL）——一种用于脉冲神经网络的互补学习规则，它利用脉冲时序统计量提取并修正有效往复连接之间的不对称性。除具备脉冲驱动与完全局部的特性外，我们的机制还主动利用噪声优势。基于赫布型与反赫布型可塑性的相互作用，突触能够恢复真实的局部梯度，并缓解神经元与突触变异性（物理神经网络中普遍存在的特性）所导致的偏差。我们通过不同脉冲网络模型验证了该机制的有效性。首先，相较于仅依赖赫布可塑性，SAL能显著提升概率脉冲网络向目标分布的收敛性。其次，在基于皮层微回路的神经层级中，SAL有效将反馈权重与前向通路对齐，从而支持正确反馈误差的反向传播。第三，我们的方法仅利用局部可塑性即可在深度网络中实现具有竞争力的性能表现。