Motivated by the widespread use of approximate derivatives in machine learning and optimization, we study inexact subgradient methods with non-vanishing additive errors and step sizes. In the nonconvex semialgebraic setting, under boundedness assumptions, we prove that the method provides points that eventually fluctuate close to the critical set at a distance proportional to $\epsilon^\rho$ where $\epsilon$ is the error in subgradient evaluation and $\rho$ relates to the geometry of the problem. In the convex setting, we provide complexity results for the averaged values. We also obtain byproducts of independent interest, such as descent-like lemmas for nonsmooth nonconvex problems and some results on the limit of affine interpolants of differential inclusions.
翻译:受机器学习与优化中广泛使用近似导数的启发,本文研究具有非消失加性误差和步长的非精确次梯度方法。在非凸半代数设定下,基于有界性假设,我们证明该方法提供的点最终会在临界点集附近波动,波动距离与$\epsilon^\rho$成比例,其中$\epsilon$为次梯度评估误差,$\rho$与问题的几何结构相关。在凸设定下,我们给出平均值方法的复杂度结果。此外,我们还获得若干具有独立价值的副产品,例如针对非光滑非凸问题的下降型引理,以及关于微分包含的仿射插值极限的一些结论。