A "dark cloud" hangs over numerical optimization theory for decades, namely, whether an optimization algorithm $O(\log(n))$ iteration complexity exists. "Yes", this paper answers, with a new optimization algorithm and strict theory proof. It starts with box-constrained quadratic programming (Box-QP), and many practical optimization problems fall into Box-QP. Smooth quadratic programming (QP) and nonsmooth Lasso can be reformulated as Box-QP via duality theory. It is the first time to present an $O(\log(n))$ iteration complexity QP algorithm, in particular, which behaves like a "direct" method: the required number of iterations is deterministic with exact value $\left\lceil\log\left(\frac{3.125n}{\epsilon}\right)/\log(1.5625)\right\rceil$. This significant breakthrough enables us to transition from the $O(\sqrt{n})$ to the $O(\log(n))$ optimization algorithm, whose amazing scalability is particularly relevant in today's era of big data and artificial intelligence.
翻译:数值优化理论数十年来笼罩着一片"阴云",即是否存在迭代复杂度为$O(\log(n))$的优化算法。本文给出了肯定答案,提出了一种新型优化算法并给出了严格的理论证明。该算法从箱型约束二次规划(Box-QP)入手,而许多实际优化问题均可归约为Box-QP。通过对偶理论,光滑二次规划(QP)和非光滑Lasso问题均可转化为Box-QP形式。本文首次提出迭代复杂度为$O(\log(n))$的QP算法,其表现如同"直接"法:所需迭代次数具有确定性精确值$\left\lceil\log\left(\frac{3.125n}{\epsilon}\right)/\log(1.5625)\right\rceil$。这一重大突破使我们能够从$O(\sqrt{n})$优化算法跨越到$O(\log(n))$优化算法,其惊人的可扩展性在当今大数据与人工智能时代具有特殊意义。