We exploit analogies between first-order algorithms for constrained optimization and non-smooth dynamical systems to design a new class of accelerated first-order algorithms for constrained optimization. Unlike Frank-Wolfe or projected gradients, these algorithms avoid optimization over the entire feasible set at each iteration. We prove convergence to stationary points even in a nonconvex setting and we derive accelerated rates for the convex setting both in continuous time, as well as in discrete time. An important property of these algorithms is that constraints are expressed in terms of velocities instead of positions, which naturally leads to sparse, local and convex approximations of the feasible set (even if the feasible set is nonconvex). Thus, the complexity tends to grow mildly in the number of decision variables and in the number of constraints, which makes the algorithms suitable for machine learning applications. We apply our algorithms to a compressed sensing and a sparse regression problem, showing that we can treat nonconvex $\ell^p$ constraints ($p<1$) efficiently, while recovering state-of-the-art performance for $p=1$.

翻译：我们利用约束优化的一阶算法与非光滑动力系统之间的类比关系，设计了一类用于约束优化的新型加速一阶算法。与Frank-Wolfe算法或投影梯度法不同，这些算法无需在每次迭代中对整个可行集进行优化。我们证明了即使在非凸场景下算法也能收敛到驻点，并在连续时间和离散时间两种框架下推导出凸场景下的加速收敛速率。这类算法的一个重要特性是将约束条件表达为速度而非位置的函数，从而自然得到可行集的稀疏、局部且凸的近似（即使可行集本身是非凸的）。因此，算法复杂度随决策变量数量及约束数量增长缓慢，使其适用于机器学习应用。我们将所提出的算法应用于压缩感知和稀疏回归问题，结果表明：我们既能高效处理非凸的$\ell^p$范数约束（$p<1$），同时在$p=1$的情况下也能恢复当前最优性能。