Neural networks are a group of neurons stacked together in multiple layers to mimic the biological neurons in a human brain. Neural networks have been trained using the backpropagation algorithm based on gradient descent strategy for several decades. Several variants have been developed to improve the backpropagation algorithm. The loss function for the neural network is optimized through backpropagation, but several local minima exist in the manifold of the constructed neural network. We obtain several solutions matching the minima. The gradient descent strategy cannot avoid the problem of local minima and gets stuck in the minima due to the initialization. Particle swarm optimization (PSO) was proposed to select the best local minima among the search space of the loss function. The search space is limited to the instantiated particles in the PSO algorithm, and sometimes it cannot select the best solution. In the proposed approach, we overcome the problem of gradient descent and the limitation of the PSO algorithm by training individual neurons separately, capable of collectively solving the problem as a group of neurons forming a network.

翻译：神经网络是一种通过多层堆叠神经元来模拟人脑中生物神经元的计算模型。数十年来，神经网络一直采用基于梯度下降策略的反向传播算法进行训练，并已发展出多种改进变体。虽然神经网络的损失函数通过反向传播进行优化，但在所构建的神经网络流形中存在多个局部极小值，我们可获得与这些极小值匹配的多种解。梯度下降策略无法规避局部极小值问题，且容易因初始化陷入极小值。粒子群优化算法曾被提出用于在损失函数搜索空间中选择最优局部极小值，但其搜索空间受限于算法实例化的粒子，有时无法选择最优解。本研究提出的方法通过对单个神经元进行独立训练，使其能够作为网络中的神经元群体协同解决问题，从而克服了梯度下降的缺陷与粒子群优化算法的局限性。