We show that any memory-constrained, first-order algorithm which minimizes $d$-dimensional, $1$-Lipschitz convex functions over the unit ball to $1/\mathrm{poly}(d)$ accuracy using at most $d^{1.25 - \delta}$ bits of memory must make at least $\tilde{\Omega}(d^{1 + (4/3)\delta})$ first-order queries (for any constant $\delta \in [0, 1/4]$). Consequently, the performance of such memory-constrained algorithms are a polynomial factor worse than the optimal $\tilde{O}(d)$ query bound for this problem obtained by cutting plane methods that use $\tilde{O}(d^2)$ memory. This resolves a COLT 2019 open problem of Woodworth and Srebro.
翻译:我们证明,任何内存受限的一阶算法,若要在单位球上最小化$d$维、$1$-利普希茨凸函数至$1/\mathrm{poly}(d)$精度,且最多使用$d^{1.25 - \delta}$比特内存(对于任意常数$\delta \in [0, 1/4]$),则必须至少进行$\tilde{\Omega}(d^{1 + (4/3)\delta})$次一阶查询。因此,此类内存受限算法的性能较最优的$\tilde{O}(d)$查询界(由使用$\tilde{O}(d^2)$内存的割平面法获得)存在多项式级差距。这解决了Woodworth与Srebro在COLT 2019年提出的公开问题。