Supervised learning in deep neural networks is commonly performed using error backpropagation. However, the sequential propagation of errors during the backward pass limits its scalability and applicability to low-powered neuromorphic hardware. Therefore, there is growing interest in finding local alternatives to backpropagation. Recently proposed methods based on forward-mode automatic differentiation suffer from high variance in large deep neural networks, which affects convergence. In this paper, we propose the Forward Direct Feedback Alignment algorithm that combines Activity-Perturbed Forward Gradients with Direct Feedback Alignment and momentum. We provide both theoretical proofs and empirical evidence that our proposed method achieves lower variance than forward gradient techniques. In this way, our approach enables faster convergence and better performance when compared to other local alternatives to backpropagation and opens a new perspective for the development of online learning algorithms compatible with neuromorphic systems.

翻译：深度神经网络中的监督学习通常采用误差反向传播算法。然而，反向传播过程中误差的顺序传播限制了其在低功耗神经形态硬件上的可扩展性和适用性。因此，寻找反向传播的局部替代方案日益受到关注。近期基于前向模式自动微分的方法在大规模深度神经网络中面临高方差问题，这影响了算法收敛性。本文提出前向直接反馈对齐算法，该算法将活动扰动前向梯度与直接反馈对齐及动量相结合。我们通过理论证明和实验证据表明，所提方法较前向梯度技术具有更低的方差。相较于其他反向传播的局部替代方案，我们的方法能够实现更快的收敛速度和更优的性能，为开发与神经形态系统兼容的在线学习算法开辟了新视角。