This paper presents a new methodology to alleviate the fundamental trade-off between accuracy and latency in spiking neural networks (SNNs). The approach involves decoding confidence information over time from the SNN outputs and using it to develop a decision-making agent that can dynamically determine when to terminate each inference. The proposed method, Dynamic Confidence, provides several significant benefits to SNNs. 1. It can effectively optimize latency dynamically at runtime, setting it apart from many existing low-latency SNN algorithms. Our experiments on CIFAR-10 and ImageNet datasets have demonstrated an average 40% speedup across eight different settings after applying Dynamic Confidence. 2. The decision-making agent in Dynamic Confidence is straightforward to construct and highly robust in parameter space, making it extremely easy to implement. 3. The proposed method enables visualizing the potential of any given SNN, which sets a target for current SNNs to approach. For instance, if an SNN can terminate at the most appropriate time point for each input sample, a ResNet-50 SNN can achieve an accuracy as high as 82.47% on ImageNet within just 4.71 time steps on average. Unlocking the potential of SNNs needs a highly-reliable decision-making agent to be constructed and fed with a high-quality estimation of ground truth. In this regard, Dynamic Confidence represents a meaningful step toward realizing the potential of SNNs.

翻译：本文提出了一种新方法，以缓解脉冲神经网络（SNN）中精度与延迟之间的根本性权衡。该方法通过从SNN输出中随时间解码置信度信息，并利用其构建决策智能体，从而动态确定每次推理的终止时机。所提出的动态置信度方法为SNN带来了多项显著优势：1）它能够在运行时有效动态优化延迟，这与许多现有低延迟SNN算法不同。我们在CIFAR-10和ImageNet数据集上的实验表明，应用动态置信度后，八种不同设置下的平均加速比达40%。2）动态置信度中的决策智能体构造简单且在参数空间具有高度鲁棒性，因此极易实现。3）该方法能够可视化任意给定SNN的潜力，为当前SNN设定了可逼近的目标。举例而言，若SNN能为每个输入样本在最优时间点终止推理，则ResNet-50 SNN在ImageNet上仅需平均4.71个时间步即可实现高达82.47%的精度。释放SNN的潜力需要构建高度可靠的决策智能体，并将其输入高质量的标签真值估计。在此方面，动态置信度是迈向实现SNN潜力的重要一步。