The Backpropagation algorithm has often been criticised for its lack of biological realism. In an attempt to find a more biologically plausible alternative, the recently introduced Forward-Forward algorithm replaces the forward and backward passes of Backpropagation with two forward passes. In this work, we show that the internal representations obtained by the Forward-Forward algorithm can organise into category-specific ensembles exhibiting high sparsity -- composed of a low number of active units. This situation is reminiscent of what has been observed in cortical sensory areas, where neuronal ensembles are suggested to serve as the functional building blocks for perception and action. Interestingly, while this sparse pattern does not typically arise in models trained with standard Backpropagation, it can emerge in networks trained with Backpropagation on the same objective proposed for the Forward-Forward algorithm.

翻译：反向传播算法常因缺乏生物合理性而受到批评。为寻找更具生物可信度的替代方案，近期提出的前向-前向算法用两个前向传递取代了反向传播的前向与反向传递。本研究发现，通过前向-前向算法获得的内部表征能够自组织成具有高度稀疏性（即少量活跃单元构成）的类别特异性集成。这种现象与在皮层感觉区观察到的情形相似——神经元集成被认为是感知与行动的功能构建模块。值得注意的是，虽然这种稀疏模式在标准反向传播训练模型中通常不会出现，但在使用相同目标函数（专为前向-前向算法设计）进行反向传播训练的网络中同样可能涌现。