Currently, neural-network processing in machine learning applications relies on layer synchronization, whereby neurons in a layer aggregate incoming currents from all neurons in the preceding layer, before evaluating their activation function. This is practiced even in artificial Spiking Neural Networks (SNNs), which are touted as consistent with neurobiology, in spite of processing in the brain being, in fact asynchronous. A truly asynchronous system however would allow all neurons to evaluate concurrently their threshold and emit spikes upon receiving any presynaptic current. Omitting layer synchronization is potentially beneficial, for latency and energy efficiency, but asynchronous execution of models previously trained with layer synchronization may entail a mismatch in network dynamics and performance. We present a study that documents and quantifies this problem in three datasets on our simulation environment that implements network asynchrony, and we show that models trained with layer synchronization either perform sub-optimally in absence of the synchronization, or they will fail to benefit from any energy and latency reduction, when such a mechanism is in place. We then "make ends meet" and address the problem with unlayered backprop, a novel backpropagation-based training method, for learning models suitable for asynchronous processing. We train with it models that use different neuron execution scheduling strategies, and we show that although their neurons are more reactive, these models consistently exhibit lower overall spike density (up to 50%), reach a correct decision faster (up to 2x) without integrating all spikes, and achieve superior accuracy (up to 10% higher). Our findings suggest that asynchronous event-based (neuromorphic) AI computing is indeed more efficient, but we need to seriously rethink how we train our SNN models, to benefit from it.

翻译：目前，机器学习应用中的神经网络处理依赖于层同步机制，即同一层中的神经元在评估激活函数之前，会聚合来自前一层所有神经元的输入电流。这种做法甚至被应用于人工脉冲神经网络（SNNs）中——尽管大脑的处理本质上是异步的，但SNNs仍被标榜为与神经生物学原理一致。然而，一个真正的异步系统将允许所有神经元在接收到任何突触前电流时，并发地评估其阈值并发放脉冲。省略层同步机制在降低延迟和提高能效方面具有潜在优势，但对于先前在层同步条件下训练的模型，异步执行可能导致网络动态与性能失配。本研究通过在我们实现网络异步的仿真环境中，对三个数据集进行问题记录与量化分析，结果表明：在层同步机制下训练的模型，若在无同步条件下运行，其性能表现欠佳；若保留该机制，则无法获得任何能效与延迟的改善。为此，我们提出"两端兼顾"的解决方案，采用一种基于反向传播的新型训练方法——无分层反向传播，以学习适用于异步处理的模型。我们利用该方法训练了采用不同神经元执行调度策略的模型，实验证明：尽管这些模型的神经元具有更高的反应性，但其整体脉冲密度持续降低（最高达50%），无需整合全部脉冲即可更快达成正确决策（速度提升最高达2倍），并实现了更高的准确率（提升最高达10%）。我们的研究结果表明，基于事件的异步（神经形态）AI计算确实具有更高效率，但为了充分发挥其优势，我们必须从根本上重新思考SNN模型的训练方法。