The recent successes in analyzing images with deep neural networks are almost exclusively achieved with Convolutional Neural Networks (CNNs). The training of these CNNs, and in fact of all deep neural network architectures, uses the backpropagation algorithm where the output of the network is compared with the desired result and the difference is then used to tune the weights of the network towards the desired outcome. In a 2022 preprint, Geoffrey Hinton suggested an alternative way of training which passes the desired results together with the images at the input of the network. This so called Forward Forward (FF) algorithm has up to now only been used in fully connected networks. In this paper, we show how the FF paradigm can be extended to CNNs. Our FF-trained CNN, featuring a novel spatially-extended labeling technique, achieves a classification accuracy of 99.0% on the MNIST hand-written digits dataset. We show how different hyperparameters affect the performance of the proposed algorithm and compare the results with CNN trained with the standard backpropagation approach. Furthermore, we use Class Activation Maps to investigate which type of features are learnt by the FF algorithm.

翻译：近期利用深度神经网络分析图像所取得的成功几乎全部归功于卷积神经网络（CNN）。这些CNN（以及所有深度神经网络架构）的训练均采用反向传播算法：将网络输出与期望结果进行比较，再利用二者的差异调整网络权重以趋近目标。在2022年的一篇预印本中，杰弗里·辛顿提出了一种替代训练方法，该方法将期望结果与图像一同输入网络。这种被称为Forward-Forward（FF）算法的方案迄今仅在全连接网络中应用。本文展示了如何将FF范式扩展至CNN。我们提出的采用新型空间扩展标记技术的FF训练CNN，在MNIST手写数字数据集上实现了99.0%的分类准确率。我们研究了不同超参数对所提算法性能的影响，并将结果与采用标准反向传播方法训练的CNN进行了对比。此外，我们利用类别激活图探究了FF算法所学习到的特征类型。