Recent efforts to improve the efficiency of neuromorphic and machine learning systems have centred on developing of specialised hardware for neural networks. These systems typically feature architectures that go beyond the von Neumann model employed in general-purpose hardware such as GPUs, offering potential efficiency and performance gains. However, neural networks developed for specialised hardware must consider its specific characteristics. This requires novel training algorithms and accurate hardware models, since they cannot be abstracted as a general-purpose computing platform. In this work, we present a bottom-up approach to training neural networks for hardware-based spiking neurons and synapses, built using ferroelectric capacitors (FeCAPs) and resistive random-access memories (RRAMs), respectively. Unlike the common approach of designing hardware to fit abstract neuron or synapse models, we start with compact models of the physical device to model the computational primitives. Based on these models, we have developed a training algorithm (BRUNO) that can reliably train the networks, even when applying hardware limitations, such as stochasticity or low bit precision. We analyse and compare BRUNO with Backpropagation Through Time. We test it on different spatio-temporal datasets. First on a music prediction dataset, where a network composed of ferroelectric leaky integrate-and-fire (FeLIF) neurons is used to predict at each time step the next musical note that should be played. The second dataset consists on the classification of the Braille letters using a network composed of quantised RRAM synapses and FeLIF neurons. The performance of this network is then compared with that of networks composed of LIF neurons. Experimental results show the potential advantages of using BRUNO by reducing the time and memory required to detect spatio-temporal patterns with quantised synapses.

翻译：近期提升神经形态与机器学习系统效率的研究主要聚焦于开发面向神经网络的专用硬件。此类系统通常采用超越通用硬件（如GPU）中冯·诺依曼模型的架构，具备潜在的效率与性能优势。然而，为专用硬件开发的神经网络必须考量其具体特性。由于无法将专用硬件抽象为通用计算平台，这需要创新的训练算法与精确的硬件建模。本研究提出一种自底向上的方法，用于训练基于硬件的脉冲神经元与突触神经网络，其中神经元与突触分别采用铁电电容器（FeCAPs）与阻变存储器（RRAMs）构建。不同于常见的为抽象神经元或突触模型设计硬件的思路，我们从物理器件的紧凑模型出发，对计算基元进行建模。基于这些模型，我们开发了一种训练算法（BRUNO），即使在施加随机性或低比特精度等硬件限制条件下，仍能可靠地训练网络。我们分析并比较了BRUNO与时间反向传播算法，并在不同时空数据集上进行了测试。首先在音乐预测数据集上，使用由铁电泄漏积分发放（FeLIF）神经元构成的网络，在每个时间步预测下一个应演奏的音符。第二个数据集涉及布莱叶字母分类任务，采用由量化RRAM突触与FeLIF神经元构成的网络进行处理，并将其性能与LIF神经元构成的网络进行对比。实验结果表明，BRUNO算法通过降低量化突触时空模式检测所需的时间与内存，展现出潜在的应用优势。