In this paper, we use composite optimization algorithms to solve sigmoid networks. We equivalently transfer the sigmoid networks to a convex composite optimization and propose the composite optimization algorithms based on the linearized proximal algorithms and the alternating direction method of multipliers. Under the assumptions of the weak sharp minima and the regularity condition, the algorithm is guaranteed to converge to a globally optimal solution of the objective function even in the case of non-convex and non-smooth problems. Furthermore, the convergence results can be directly related to the amount of training data and provide a general guide for setting the size of sigmoid networks. Numerical experiments on Franke's function fitting and handwritten digit recognition show that the proposed algorithms perform satisfactorily and robustly.

翻译：本文采用复合优化算法求解Sigmoid网络。我们将Sigmoid网络等价转化为凸复合优化问题，并基于线性化近端算法和交替方向乘子法提出了复合优化算法。在弱尖锐极小值和正则性条件假设下，该算法即使在非凸非光滑问题中也能保证收敛到目标函数的全局最优解。进一步地，收敛结果可直接关联训练数据量，并为设定Sigmoid网络规模提供通用指导。Franke函数拟合与手写数字识别的数值实验表明，所提算法性能优良且具有鲁棒性。