The Backpropagation algorithm, widely used to train neural networks, has often been criticised for its lack of biological realism. In an attempt to find a more biologically plausible alternative, and avoid to back-propagate gradients in favour of using local learning rules, the recently introduced Forward-Forward algorithm replaces the traditional forward and backward passes of Backpropagation with two forward passes. In this work, we show that internal representations obtained with the Forward-Forward algorithm organize into robust, category-specific ensembles, composed by an extremely low number of active units (high sparsity). This is remarkably similar to what is observed in cortical representations during sensory processing. While not found in models trained with standard Backpropagation, sparsity emerges also in networks optimized by Backpropagation, on the same training objective of Forward-Forward. These results suggest that the learning procedure proposed by Forward-Forward may be superior to Backpropagation in modelling learning in the cortex, even when a backward pass is used.

翻译：反向传播算法作为训练神经网络的常用方法，常因缺乏生物合理性而受到批评。为探索更具生物可能性的替代方案并避免梯度反向传播，转而采用局部学习规则，近期提出的前向-前向算法将反向传播的传统前向与反向传递替换为两次前向传递。本研究表明，通过前向-前向算法获得的内部表征会组织成鲁棒的、类别特定的整体，其由极少数活跃单元（高稀疏性）构成。这与感觉处理过程中皮层表征的观察结果极为相似。尽管标准反向传播训练的模型中未发现此现象，但在使用与前后向传播相同训练目标进行优化的反向传播网络中，稀疏性同样涌现。这些结果表明，前向-前向算法提出的学习过程在模拟皮层学习方面可能优于反向传播——即便后者采用了反向传递。