The joint progress of artificial neural networks (ANNs) and domain specific hardware accelerators such as GPUs and TPUs took over many domains of machine learning research. This development is accompanied by a rapid growth of the required computational demands for larger models and more data. Concurrently, emerging properties of foundation models such as in-context learning drive new opportunities for machine learning applications. However, the computational cost of such applications is a limiting factor of the technology in data centers, and more importantly in mobile devices and edge systems. To mediate the energy footprint and non-trivial latency of contemporary systems, neuromorphic computing systems deeply integrate computational principles of neurobiological systems by leveraging low-power analog and digital technologies. SpiNNaker2 is a digital neuromorphic chip developed for scalable machine learning. The event-based and asynchronous design of SpiNNaker2 allows the composition of large-scale systems involving thousands of chips. This work features the operating principles of SpiNNaker2 systems, outlining the prototype of novel machine learning applications. These applications range from ANNs over bio-inspired spiking neural networks to generalized event-based neural networks. With the successful development and deployment of SpiNNaker2, we aim to facilitate the advancement of event-based and asynchronous algorithms for future generations of machine learning systems.
翻译:人工神经网络(ANN)与GPU、TPU等专用硬件加速器的协同发展,已主导了机器学习研究的诸多领域。这一进展伴随着模型规模扩大与数据量增长所引发的计算需求急剧攀升。与此同时,基础模型涌现出的上下文学习等新特性,为机器学习应用带来了新的机遇。然而,这类应用的计算成本成为数据中心技术的关键制约因素,在移动设备与边缘系统中尤为突出。为了缓解当代系统的能耗负担与非平凡延迟,神经形态计算系统通过采用低功耗模拟与数字技术,深度融合了神经生物系统的计算原理。SpiNNaker2是一款面向可扩展机器学习设计的数字神经形态芯片。其基于事件驱动与异步架构的设计,使得包含数千芯片的大规模系统集成成为可能。本文阐述了SpiNNaker2系统的工作原理,勾勒了新型机器学习应用的原型——这些应用涵盖从人工神经网络、生物启发脉冲神经网络,到广义事件驱动神经网络。通过成功研发并部署SpiNNaker2,我们旨在推动面向未来机器学习系统的事件驱动与异步算法发展。