Backpropagation has long been criticized for being biologically implausible due to its reliance on concepts that are not viable in natural learning processes. Two core issues are the weight transport and update locking problems caused by the forward-backward dependencies, which limit biological plausibility, computational efficiency, and parallelization. Although several alternatives have been proposed to increase biological plausibility, they often come at the cost of reduced predictive performance. This paper proposes an alternative approach to training feed-forward neural networks addressing these issues by using approximate gradient information. We introduce Feed-Forward with delayed Feedback (F$^3$), which approximates gradients using fixed random feedback paths and delayed error information from the previous epoch to balance biological plausibility with predictive performance. We evaluate F$^3$ across multiple tasks and architectures, including both fully-connected and Transformer networks. Our results demonstrate that, compared to similarly plausible approaches, F$^3$ significantly improves predictive performance, narrowing the gap to backpropagation by up to 56% for classification and 96% for regression. This work is a step towards more biologically plausible learning algorithms while opening up new avenues for energy-efficient and parallelizable neural network training.

翻译：反向传播因其依赖自然学习过程中不可行的概念，长期以来被批评为生物学上不可信。两个核心问题是由前向-后向依赖引起的权重传输和更新锁定问题，这些问题限制了生物学可信度、计算效率和并行化能力。尽管已提出多种替代方案以增强生物学可信度，但它们通常以预测性能下降为代价。本文提出一种替代方法，通过使用近似梯度信息来训练前馈神经网络以解决这些问题。我们引入具有延迟反馈的前馈网络（F$^3$），该方法利用固定的随机反馈路径和来自前一训练周期的延迟误差信息来近似梯度，从而在生物学可信度与预测性能之间取得平衡。我们在多种任务和架构上评估F$^3$，包括全连接网络和Transformer网络。实验结果表明，与具有相似可信度的方案相比，F$^3$显著提升了预测性能，将分类任务中与反向传播的性能差距缩小了56%，回归任务中缩小了96%。这项工作为开发更具生物学可信度的学习算法迈出了重要一步，同时为节能且可并行的神经网络训练开辟了新途径。