This paper introduces a general framework for iterative optimization algorithms and establishes under general assumptions that their convergence is asymptotically geometric. We also prove that under appropriate assumptions, the rate of convergence can be lower bounded. The convergence is then only geometric, and we provide the exact asymptotic convergence rate. This framework allows to deal with constrained optimization and encompasses the Expectation Maximization algorithm and the mirror descent algorithm, as well as some variants such as the alpha-Expectation Maximization or the Mirror Prox algorithm.Furthermore, we establish sufficient conditions for the convergence of the Mirror Prox algorithm, under which the method converges systematically to the unique minimizer of a convex function on a convex compact set.
翻译:本文提出了一种迭代优化算法的通用框架,并在一般性假设下证明其收敛具有渐近几何性。我们进一步证明,在适当假设下,收敛速度存在下界。此时收敛仅为几何级数,并给出了精确的渐近收敛速率。该框架可处理约束优化问题,涵盖期望最大化算法与镜像下降算法,以及α-期望最大化或镜像近端算法等变体。此外,我们建立了镜像近端算法收敛的充分条件,在该条件下方法能系统性收敛到凸紧集上凸函数的唯一极小点。