In this paper, we use composite optimization algorithms to solve sigmoid networks. We equivalently transfer the sigmoid networks to a convex composite optimization and propose the composite optimization algorithms based on the linearized proximal algorithms and the alternating direction method of multipliers. Under the assumptions of the weak sharp minima and the regularity condition, the algorithm is guaranteed to converge to a globally optimal solution of the objective function even in the case of non-convex and non-smooth problems. Furthermore, the convergence results can be directly related to the amount of training data and provide a general guide for setting the size of sigmoid networks. Numerical experiments on Franke's function fitting and handwritten digit recognition show that the proposed algorithms perform satisfactorily and robustly.

翻译：本文采用复合优化算法求解Sigmoid网络。我们将Sigmoid网络等价转化为凸复合优化问题，并基于线性化近端算法和交替方向乘子法提出复合优化算法。在弱尖锐极小值条件和正则性假设下，即使面对非凸非光滑问题，该算法也能保证收敛至目标函数的全局最优解。此外，收敛结果可直接关联训练数据量，为设定Sigmoid网络的规模提供通用指导。Franke函数拟合与手写数字识别的数值实验表明，所提算法表现优异且具有鲁棒性。