We exploit analogies between first-order algorithms for constrained optimization and non-smooth dynamical systems to design a new class of accelerated first-order algorithms for constrained optimization. Unlike Frank-Wolfe or projected gradients, these algorithms avoid optimization over the entire feasible set at each iteration. We prove convergence to stationary points even in a nonconvex setting and we derive rates for the convex setting. An important property of these algorithms is that constraints are expressed in terms of velocities instead of positions, which naturally leads to sparse, local and convex approximations of the feasible set (even if the feasible set is nonconvex). Thus, the complexity tends to grow mildly in the number of decision variables and in the number of constraints, which makes the algorithms suitable for machine learning applications. We apply our algorithms to a compressed sensing and a sparse regression problem, showing that we can treat nonconvex $\ell^p$ constraints ($p<1$) efficiently, while recovering state-of-the-art performance for $p=1$.

翻译：我们利用约束优化中的一阶算法与非光滑动力系统之间的类比关系，设计了一类新的用于约束优化的加速一阶算法。与Frank-Wolfe方法或投影梯度法不同，这些算法无需在每次迭代中对整个可行集进行优化。我们证明了即使在非凸设定下，算法也能收敛至稳定点，并在凸设定下推导出收敛速率。这些算法的一个重要特性是，约束以速度而非位置的形式表达，这自然导致可行集（即便其本身是非凸的）的稀疏、局部且凸的近似。因此，计算复杂度随决策变量数量和约束数量增长较为平缓，使算法适用于机器学习应用。我们将该算法应用于压缩感知和稀疏回归问题，表明我们能高效处理非凸的$\ell^p$约束（$p<1$），同时在$p=1$情况下恢复出与现有最优方法相当的性能。