As spiking neural networks receive more attention, we look toward applications of this computing paradigm in fields other than computer vision and signal processing. One major field, underexplored in the neuromorphic setting, is Natural Language Processing (NLP), where most state-of-the-art solutions still heavily rely on resource-consuming and power-hungry traditional deep learning architectures. Therefore, it is compelling to design NLP models for neuromorphic architectures due to their low energy requirements, with the additional benefit of a more human-brain-like operating model for processing information. However, one of the biggest issues with bringing NLP to the neuromorphic setting is in properly encoding text into a spike train so that it can be seamlessly handled by both current and future SNN architectures. In this paper, we compare various methods of encoding text as spikes and assess each method's performance in an associated SNN on a downstream NLP task, namely, sentiment analysis. Furthermore, we go on to propose a new method of encoding text as spikes that outperforms a widely-used rate-coding technique, Poisson rate-coding, by around 13\% on our benchmark NLP tasks. Subsequently, we demonstrate the energy efficiency of SNNs implemented in hardware for the sentiment analysis task compared to traditional deep neural networks, observing an energy efficiency increase of more than 32x during inference and 60x during training while incurring the expected energy-performance tradeoff.

翻译：随着脉冲神经网络受到更多关注，我们开始探索该计算范式在计算机视觉和信号处理之外领域的应用。一个在神经形态计算中尚未充分探索的重要领域是自然语言处理（NLP），其中大多数最先进的解决方案仍严重依赖资源消耗大且功耗高的传统深度学习架构。因此，为神经形态架构设计NLP模型具有重要价值，这既源于其低能耗需求，还因该模型更接近人脑处理信息的方式。然而，将NLP引入神经形态计算面临的最大挑战之一，是如何将文本正确编码为脉冲序列，使其能被当前及未来的SNN架构无缝处理。本文比较了多种文本脉冲编码方法，并在下游NLP任务（即情感分析）中评估了每种方法在关联SNN上的性能。此外，我们提出了一种新的文本脉冲编码方法，在基准NLP任务上比广泛使用的速率编码技术（泊松速率编码）性能提升约13%。随后，我们展示了在硬件中实现的SNN在情感分析任务上相较于传统深度神经网络的能效优势，观察到推理阶段能效提升超过32倍，训练阶段超过60倍，同时伴随预期中的能效-性能权衡。