In this paper, we use composite optimization algorithms to solve sigmoid networks. We equivalently transfer the sigmoid networks to a convex composite optimization and propose the composite optimization algorithms based on the linearized proximal algorithms and the alternating direction method of multipliers. Under the assumptions of the weak sharp minima and the regularity condition, the algorithm is guaranteed to converge to a globally optimal solution of the objective function even in the case of non-convex and non-smooth problems. Furthermore, the convergence results can be directly related to the amount of training data and provide a general guide for setting the size of sigmoid networks. Numerical experiments on Franke's function fitting and handwritten digit recognition show that the proposed algorithms perform satisfactorily and robustly.

翻译：本文采用复合优化算法求解Sigmoid网络。我们将Sigmoid网络等价转化为凸复合优化问题，并基于线性化近端算法和交替方向乘子法提出了复合优化算法。在弱锐极小值和正则性条件的假设下，即使目标函数非凸且非光滑，该算法也能保证收敛至全局最优解。此外，收敛结果可直接关联训练数据量，为Sigmoid网络规模设置提供通用指导。针对Franke函数拟合和手写数字识别的数值实验表明，所提算法性能优越且具有鲁棒性。