Recurrent neural networks (RNNs) hold immense potential for computations due to their Turing completeness and sequential processing capabilities, yet existing methods for their training encounter efficiency challenges. Backpropagation through time (BPTT), the prevailing method, extends the backpropagation (BP) algorithm by unrolling the RNN over time. However, this approach suffers from significant drawbacks, including the need to interleave forward and backward phases and store exact gradient information. Furthermore, BPTT has been shown to struggle to propagate gradient information for long sequences, leading to vanishing gradients. An alternative strategy to using gradient-based methods like BPTT involves stochastically approximating gradients through perturbation-based methods. This learning approach is exceptionally simple, necessitating only forward passes in the network and a global reinforcement signal as feedback. Despite its simplicity, the random nature of its updates typically leads to inefficient optimization, limiting its effectiveness in training neural networks. In this study, we present a new approach to perturbation-based learning in RNNs whose performance is competitive with BPTT, while maintaining the inherent advantages over gradient-based learning. To this end, we extend the recently introduced activity-based node perturbation (ANP) method to operate in the time domain, leading to more efficient learning and generalization. We subsequently conduct a range of experiments to validate our approach. Our results show similar performance, convergence time and scalability compared to BPTT, strongly outperforming standard node and weight perturbation methods. These findings suggest that perturbation-based learning methods offer a versatile alternative to gradient-based methods for training RNNs which can be ideally suited for neuromorphic computing applications

翻译：循环神经网络（RNN）因其图灵完备性和序列处理能力在计算领域具有巨大潜力，但现有训练方法面临效率挑战。随时间反向传播（BPTT）作为主流方法，通过沿时间维度展开RNN来扩展反向传播（BP）算法。然而该方法存在明显缺陷：需要交替执行前向与后向传播阶段，且必须存储精确的梯度信息。此外，研究已表明BPTT难以在长序列中有效传递梯度信息，导致梯度消失问题。替代梯度类方法（如BPTT）的一种策略是通过基于扰动的方法进行随机梯度近似。这种学习机制异常简洁，仅需网络前向传播和全局强化信号作为反馈。尽管结构简单，但其随机更新特性通常导致优化效率低下，限制了在神经网络训练中的有效性。本研究提出一种新型的RNN扰动学习方法，其性能可与BPTT相媲美，同时保持相对于梯度学习方法的固有优势。为此，我们将近期提出的基于活动的节点扰动（ANP）方法扩展至时间域，实现了更高效的学习与泛化能力。随后通过系列实验验证该方法，结果表明：在性能、收敛时间和可扩展性方面与BPTT表现相当，并显著优于标准节点扰动与权重扰动方法。这些发现表明，基于扰动的学习方法为RNN训练提供了多功能的梯度方法替代方案，尤其适用于神经形态计算应用场景。