Convolutional Neural Networks (ConvNets or CNNs) have been candidly deployed in the scope of computer vision and related fields. Nevertheless, the dynamics of training of these neural networks lie still elusive: it is hard and computationally expensive to train them. A myriad of architectures and training strategies have been proposed to overcome this challenge and address several problems in image processing such as speech, image and action recognition as well as object detection. In this article, we propose a novel Particle Swarm Optimization (PSO) based training for ConvNets. In such framework, the vector of weights of each ConvNet is typically cast as the position of a particle in phase space whereby PSO collaborative dynamics intertwines with Stochastic Gradient Descent (SGD) in order to boost training performance and generalization. Our approach goes as follows: i) [regular phase] each ConvNet is trained independently via SGD; ii) [collaborative phase] ConvNets share among themselves their current vector of weights (or particle-position) along with their gradient estimates of the Loss function. Distinct step sizes are coined by distinct ConvNets. By properly blending ConvNets with large (possibly random) step-sizes along with more conservative ones, we propose an algorithm with competitive performance with respect to other PSO-based approaches on Cifar-10 and Cifar-100 (accuracy of 98.31% and 87.48%). These accuracy levels are obtained by resorting to only four ConvNets -- such results are expected to scale with the number of collaborative ConvNets accordingly. We make our source codes available for download https://github.com/leonlha/PSO-ConvNet-Dynamics.

翻译：卷积神经网络（ConvNets或CNNs）已被广泛应用于计算机视觉及相关领域。然而，这些神经网络的训练动态仍难以捉摸：训练过程既困难又计算成本高昂。为应对这一挑战并解决图像处理中的诸多问题（如语音识别、图像识别、动作识别及目标检测），研究者提出了大量网络架构与训练策略。本文提出了一种基于粒子群优化（PSO）的新型ConvNet训练方法。在该框架中，每个ConvNet的权重向量被视作相空间中粒子的位置，PSO的协作动力学与随机梯度下降（SGD）相互交织，以提升训练性能与泛化能力。我们的方法具体如下：i) [常规阶段] 每个ConvNet通过SGD独立训练；ii) [协作阶段] ConvNet之间共享其当前权重向量（即粒子位置）及损失函数的梯度估计值。不同的ConvNet采用不同的步长。通过合理混合采用大（可能为随机）步长与更保守步长的ConvNet，我们提出的算法在Cifar-10和Cifar-100数据集上取得了与其他基于PSO的方法相媲美的竞争性能（准确率分别为98.31%和87.48%）。这些准确率仅通过四个ConvNet实现——预计该结果将随协作ConvNet数量的增加而相应提升。我们已将源代码公开于https://github.com/leonlha/PSO-ConvNet-Dynamics。