A novel high-fan-in differential superconductor neuron structure designed for ultra-high-performance Spiking Neural Network (SNN) accelerators is presented. Utilizing a high-fan-in neuron structure allows us to design SNN accelerators with more synaptic connections, enhancing the overall network capabilities. The proposed neuron design is based on superconductor electronics fabric, incorporating multiple superconducting loops, each with two Josephson Junctions. This arrangement enables each input data branch to have positive and negative inductive coupling, supporting excitatory and inhibitory synaptic data. Compatibility with synaptic devices and thresholding operation is achieved using a single flux quantum (SFQ) pulse-based logic style. The neuron design, along with ternary synaptic connections, forms the foundation for a superconductor-based SNN inference. To demonstrate the capabilities of our design, we train the SNN using snnTorch, augmenting the PyTorch framework. After pruning, the demonstrated SNN inference achieves an impressive 96.1% accuracy on MNIST images. Notably, the network exhibits a remarkable throughput of 8.92 GHz while consuming only 1.5 nJ per inference, including the energy consumption associated with cooling to 4K. These results underscore the potential of superconductor electronics in developing high-performance and ultra-energy-efficient neural network accelerator architectures.
翻译:本文提出了一种新型高扇入差分超导神经元结构,专为超高性能脉冲神经网络(SNN)加速器设计。采用高扇入神经元结构可构建具有更多突触连接的SNN加速器,从而增强整体网络能力。该神经元设计基于超导电子学工艺,集成多个超导环,每个环包含两个约瑟夫森结。这种架构使得每个输入数据支路能够支持正负电感耦合,从而实现兴奋性和抑制性突触数据的处理。通过采用基于单磁通量子(SFQ)脉冲的逻辑样式,实现了与突触器件及阈值操作的兼容性。该神经元设计结合三元突触连接,构成了基于超导体的SNN推理基础。为验证设计的性能,我们使用snnTorch(基于PyTorch框架扩展)训练SNN。经过剪枝后,所演示的SNN推理在MNIST图像数据集上实现了96.1%的准确率。值得注意的是,该网络展现出8.92 GHz的卓越吞吐量,且单次推理能耗仅1.5 nJ(包含冷却至4K的能耗)。这些结果凸显了超导电子学在开发高性能、超节能神经网络加速器架构方面的巨大潜力。