"Forward-only" algorithms, which train neural networks while avoiding a backward pass, have recently gained attention as a way of solving the biologically unrealistic aspects of backpropagation. Here, we first discuss the similarities between two "forward-only" algorithms, the Forward-Forward and PEPITA frameworks, and demonstrate that PEPITA is equivalent to a Forward-Forward with top-down feedback connections. Then, we focus on PEPITA to address compelling challenges related to the "forward-only" rules, which include providing an analytical understanding of their dynamics and reducing the gap between their performance and that of backpropagation. We propose a theoretical analysis of the dynamics of PEPITA. In particular, we show that PEPITA is well-approximated by an "adaptive-feedback-alignment" algorithm and we analytically track its performance during learning in a prototype high-dimensional setting. Finally, we develop a strategy to apply the weight mirroring algorithm on "forward-only" algorithms with top-down feedback and we show how it impacts PEPITA's accuracy and convergence rate.

翻译："仅前向"算法通过避免反向传播来训练神经网络，近期因解决了反向传播中生物不现实的问题而受到关注。本文首先探讨了两种"仅前向"算法——Forward-Forward框架与PEPITA框架——的相似性，并论证PEPITA等价于具有自上而下反馈连接的Forward-Forward算法。继而，我们聚焦于PEPITA，以应对与"仅前向"规则相关的若干挑战性课题，包括对其动态特性的分析性理解，以及缩小其与反向传播算法性能之间的差距。我们提出了PEPITA动态特性的理论分析框架，具体而言，证明了PEPITA可被"自适应反馈对齐"算法良好逼近，并在原型高维场景中对其学习过程中的性能进行了分析性追踪。最后，我们开发了一种将权重镜像算法应用于具有自上而下反馈的"仅前向"算法的策略，并展示了该策略对PEPITA精度与收敛速率的影响。