We introduce a new approach in distributed deep learning, utilizing Geoffrey Hinton's Forward-Forward (FF) algorithm to speed up the training of neural networks in distributed computing environments. Unlike traditional methods that rely on forward and backward passes, the FF algorithm employs a dual forward pass strategy, significantly diverging from the conventional backpropagation process. This novel method aligns more closely with the human brain's processing mechanisms, potentially offering a more efficient and biologically plausible approach to neural network training. Our research explores different implementations of the FF algorithm in distributed settings, to explore its capacity for parallelization. While the original FF algorithm focused on its ability to match the performance of the backpropagation algorithm, the parallelism aims to reduce training times and resource consumption, thereby addressing the long training times associated with the training of deep neural networks. Our evaluation shows a 3.75 times speed up on MNIST dataset without compromising accuracy when training a four-layer network with four compute nodes. The integration of the FF algorithm into distributed deep learning represents a significant step forward in the field, potentially revolutionizing the way neural networks are trained in distributed environments.

翻译：我们提出了一种分布式深度学习的新方法，利用杰弗里·辛顿提出的向前-向前（FF）算法来加速分布式计算环境中神经网络的训练。与传统依赖前向和反向传播的方法不同，FF算法采用双前向传播策略，显著区别于常规的反向传播过程。这种新颖方法更接近人脑的处理机制，可能为神经网络训练提供一种更高效且更具生物学合理性的途径。本研究探索了FF算法在分布式环境中的不同实现方式，以评估其并行化能力。原始FF算法侧重于匹配反向传播算法的性能，而并行化旨在缩短训练时间并降低资源消耗，从而解决深度神经网络训练耗时长的难题。我们的评估表明，在MNIST数据集上使用四个计算节点训练四层网络时，速度提升了3.75倍，且未牺牲准确性。将FF算法融入分布式深度学习，代表了该领域的重大进步，有望彻底改变分布式环境中神经网络的训练方式。