Backpropagation algorithm has been widely used as a mainstream learning procedure for neural networks in the past decade, and has played a significant role in the development of deep learning. However, there exist some limitations associated with this algorithm, such as getting stuck in local minima and experiencing vanishing/exploding gradients, which have led to questions about its biological plausibility. To address these limitations, alternative algorithms to backpropagation have been preliminarily explored, with the Forward-Forward (FF) algorithm being one of the most well-known. In this paper we propose a new learning framework for neural networks, namely Cascaded Forward (CaFo) algorithm, which does not rely on BP optimization as that in FF. Unlike FF, our framework directly outputs label distributions at each cascaded block, which does not require generation of additional negative samples and thus leads to a more efficient process at both training and testing. Moreover, in our framework each block can be trained independently, so it can be easily deployed into parallel acceleration systems. The proposed method is evaluated on four public image classification benchmarks, and the experimental results illustrate significant improvement in prediction accuracy in comparison with the baseline.

翻译：反向传播算法在过去十年中作为神经网络的主流学习过程被广泛使用，并在深度学习的发展中发挥了重要作用。然而，该算法存在一些局限性，例如陷入局部极小值和遭遇梯度消失/爆炸问题，这引发了对其生物合理性的质疑。为应对这些局限性，研究者初步探索了替代反向传播的算法，其中Forward-Forward（FF）算法最为知名。本文提出一种新的神经网络学习框架——级联前馈（Cascaded Forward, CaFo）算法，该算法不依赖反向传播优化。与FF不同，我们的框架直接在每个级联模块输出标签分布，无需生成额外的负样本，因此在训练和测试阶段均更为高效。此外，在该框架中，每个模块可独立训练，便于轻松部署到并行加速系统中。所提方法在四个公开图像分类基准上进行了评估，实验结果表明，与基线相比，预测精度显著提升。