Backpropagation algorithm has been widely used as a mainstream learning procedure for neural networks in the past decade, and has played a significant role in the development of deep learning. However, there exist some limitations associated with this algorithm, such as getting stuck in local minima and experiencing vanishing/exploding gradients, which have led to questions about its biological plausibility. To address these limitations, alternative algorithms to backpropagation have been preliminarily explored, with the Forward-Forward (FF) algorithm being one of the most well-known. In this paper we propose a new learning framework for neural networks, namely Cascaded Forward (CaFo) algorithm, which does not rely on BP optimization as that in FF. Unlike FF, our framework directly outputs label distributions at each cascaded block, which does not require generation of additional negative samples and thus leads to a more efficient process at both training and testing. Moreover, in our framework each block can be trained independently, so it can be easily deployed into parallel acceleration systems. The proposed method is evaluated on four public image classification benchmarks, and the experimental results illustrate significant improvement in prediction accuracy in comparison with the baseline.

翻译：反向传播算法在过去十年中作为神经网络的主流学习流程被广泛应用，并在深度学习的发展中发挥了重要作用。然而，该算法存在一些局限性，例如容易陷入局部极小值、遭遇梯度消失/爆炸问题，这引发了对其生物合理性的质疑。为克服这些局限，学界初步探索了反向传播的替代算法，其中前向-前向（Forward-Forward, FF）算法最为知名。本文提出一种新的神经网络学习框架——级联前向（Cascaded Forward, CaFo）算法，该算法不依赖反向传播优化（如FF算法所示）。与FF不同，本框架直接在每个级联块输出标签分布，无需生成额外的负样本，从而在训练和测试阶段均实现更高效的流程。此外，本框架中每个块可独立训练，因此易于部署在并行加速系统中。该方法在四个公开图像分类基准上进行评估，实验结果表明，与基线相比，预测准确率显著提升。