The Backpropagation algorithm has often been criticised for its lack of biological realism. In an attempt to find a more biologically plausible alternative, the recently introduced Forward-Forward algorithm replaces the forward and backward passes of Backpropagation with two forward passes. In this work, we show that the internal representations obtained by the Forward-Forward algorithm can organise into category-specific ensembles exhibiting high sparsity - i.e. composed of an extremely low number of active units. This situation is reminiscent of what has been observed in cortical sensory areas, where neuronal ensembles are suggested to serve as the functional building blocks for perception and action. Interestingly, while this sparse pattern does not typically arise in models trained with standard Backpropagation, it can emerge in networks trained with Backpropagation on the same objective proposed for the Forward-Forward algorithm. These results suggest that the learning procedure proposed by Forward-Forward may be superior to Backpropagation in modelling learning in the cortex, even when a backward pass is used.

翻译：反向传播算法常因其缺乏生物合理性而受到批评。为寻找更具生物学合理性的替代方案，新近提出的前向-前向算法用两次前向传播取代了反向传播的正向和反向传递。在本工作中，我们表明前向-前向算法获得的内部表征可组织成具有高度稀疏性的类别特异性集合——即由极少数活跃单元组成。这种情况令人联想到皮层感觉区域的观察结果，其中神经元集合被认为构成知觉和行动的功能性构建单元。有趣的是，虽然这种稀疏模式通常不会出现在标准反向传播训练的模型中，但基于前向-前向算法提出的同一目标函数，使用反向传播训练的网络也能涌现出这种模式。这些结果表明，前向-前向算法提出的学习过程可能在模拟皮层学习方面优于反向传播，即便在使用反向传递的情况下也是如此。