Neural networks are a group of neurons stacked together in multiple layers to mimic the biological neurons in a human brain. Neural networks have been trained using the backpropagation algorithm based on gradient descent strategy for several decades. Several variants have been developed to improve the backpropagation algorithm. The loss function for the neural network is optimized through backpropagation, but several local minima exist in the manifold of the constructed neural network. We obtain several solutions matching the minima. The gradient descent strategy cannot avoid the problem of local minima and gets stuck in the minima due to the initialization. Particle swarm optimization (PSO) was proposed to select the best local minima among the search space of the loss function. The search space is limited to the instantiated particles in the PSO algorithm, and sometimes it cannot select the best solution. In the proposed approach, we overcome the problem of gradient descent and the limitation of the PSO algorithm by training individual neurons separately, capable of collectively solving the problem as a group of neurons forming a network. Our code and data are available at https://github.com/dipkmr/train-nn-wobp/

翻译：神经网络是一种将多个神经元分层堆叠以模拟人脑中生物神经元的计算模型。数十年来，神经网络一直采用基于梯度下降策略的反向传播算法进行训练。为改进反向传播算法，已发展出多种变体。神经网络的损失函数通过反向传播进行优化，但在所构建的神经网络的流形中存在多个局部极小值。我们获得了多个与极小值匹配的解。梯度下降策略无法避免局部极小值问题，并会因初始化而陷入极小值。粒子群优化算法被提出以在损失函数的搜索空间中选择最佳局部极小值。该搜索空间受限于PSO算法中实例化的粒子，有时无法选择最优解。在本研究中，我们通过单独训练每个神经元来克服梯度下降的问题和PSO算法的局限性，这些神经元能够作为形成网络的神经元群体共同解决问题。我们的代码和数据可在 https://github.com/dipkmr/train-nn-wobp/ 获取。