A stochastic-gradient-based interior-point algorithm for minimizing a continuously differentiable objective function (that may be nonconvex) subject to bound constraints is presented, analyzed, and demonstrated through experimental results. The algorithm is unique from other interior-point methods for solving smooth (nonconvex) optimization problems since the search directions are computed using stochastic gradient estimates. It is also unique in its use of inner neighborhoods of the feasible region -- defined by a positive and vanishing neighborhood-parameter sequence -- in which the iterates are forced to remain. It is shown that with a careful balance between the barrier, step-size, and neighborhood sequences, the proposed algorithm satisfies convergence guarantees in both deterministic and stochastic settings. The results of numerical experiments show that in both settings the algorithm can outperform a projected-(stochastic)-gradient method.
翻译:提出、分析并实验验证了一种基于随机梯度的内点算法,用于最小化满足边界约束的连续可微目标函数(可能非凸)。该算法在求解光滑(非凸)优化问题时具有独特性,其搜索方向通过随机梯度估计计算。该算法另一独特之处在于使用可行域的内部邻域——由正且趋于零的邻域参数序列定义——强制迭代点始终位于该邻域内。研究表明,通过谨慎平衡罚函数、步长和邻域序列,该算法在确定性和随机设置下均满足收敛保证。数值实验结果表明,在两种设置下该算法均能优于投影(随机)梯度方法。