A novel high-fan-in differential superconductor neuron structure designed for ultra-high-performance Spiking Neural Network (SNN) accelerators is presented. Utilizing a high-fan-in neuron structure allows us to design SNN accelerators with more synaptic connections, enhancing the overall network capabilities. The proposed neuron design is based on superconductor electronics fabric, incorporating multiple superconducting loops, each with two Josephson Junctions. This arrangement enables each input data branch to have positive and negative inductive coupling, supporting excitatory and inhibitory synaptic data. Compatibility with synaptic devices and thresholding operation is achieved using a single flux quantum (SFQ) pulse-based logic style. The neuron design, along with ternary synaptic connections, forms the foundation for a superconductor-based SNN inference. To demonstrate the capabilities of our design, we train the SNN using snnTorch, augmenting the PyTorch framework. After pruning, the demonstrated SNN inference achieves an impressive 96.1% accuracy on MNIST images. Notably, the network exhibits a remarkable throughput of 8.92 GHz while consuming only 1.5 nJ per inference, including the energy consumption associated with cooling to 4K. These results underscore the potential of superconductor electronics in developing high-performance and ultra-energy-efficient neural network accelerator architectures.
翻译:本文提出了一种新颖的高扇入差分超导神经元结构,专为超高性能脉冲神经网络(SNN)加速器设计。采用高扇入神经元结构能够设计具有更多突触连接的SNN加速器,从而提升整体网络能力。所提出的神经元设计基于超导电子学工艺,包含多个超导环路,每个环路配备两个约瑟夫森结。这种设计使得每个输入数据分支具备正负电感耦合特性,支持兴奋性和抑制性突触数据。通过采用单磁通量子(SFQ)脉冲逻辑风格,实现了与突触器件的兼容性及阈值操作功能。该神经元设计结合三元突触连接,构成了基于超导体的SNN推理基础。为验证设计能力,我们采用snnTorch对PyTorch框架进行增强,对SNN进行训练。经过剪枝后,所展示的SNN推理在MNIST图像数据集上达到了96.1%的惊人准确率。值得注意的是,该网络实现了8.92 GHz的显著吞吐量,且每次推理仅消耗1.5 nJ能量(包含冷却至4K的能耗)。这些结果凸显了超导电子学在开发高性能、超节能神经网络加速器架构方面的巨大潜力。