Composite minimization is a powerful framework in large-scale convex optimization, based on decoupling of the objective function into terms with structurally different properties and allowing for more flexible algorithmic design. We introduce a new algorithmic framework for complementary composite minimization, where the objective function decouples into a (weakly) smooth and a uniformly convex term. This particular form of decoupling is pervasive in statistics and machine learning, due to its link to regularization. The main contributions of our work are summarized as follows. First, we introduce the problem of complementary composite minimization in general normed spaces; second, we provide a unified accelerated algorithmic framework to address broad classes of complementary composite minimization problems; and third, we prove that the algorithms resulting from our framework are near-optimal in most of the standard optimization settings. Additionally, we show that our algorithmic framework can be used to address the problem of making the gradients small in general normed spaces. As a concrete example, we obtain a nearly-optimal method for the standard $\ell_1$ setup (small gradients in the $\ell_{\infty}$ norm), essentially matching the bound of Nesterov (2012) that was previously known only for the Euclidean setup. Finally, we show that our composite methods are broadly applicable to a number of regression and other classes of optimization problems, where regularization plays a key role. Our methods lead to complexity bounds that are either new or match the best existing ones.

翻译：复合最小化是大规模凸优化中的一个强大框架，其核心思想是将目标函数按结构特性分解为不同项，从而允许更灵活的算法设计。我们提出了一种新的互补复合最小化算法框架，其中目标函数被分解为一个（弱）光滑项与一个一致凸项。这种分解形式因其与正则化的关联，在统计学和机器学习中普遍存在。本文的主要贡献总结如下：首先，我们引入了泛函空间中的互补复合最小化问题；其次，我们提供了一个统一的加速算法框架，用以解决广泛的互补复合最小化问题；最后，我们证明了该框架产生的算法在大多数标准优化设置中接近最优。此外，我们展示了该算法框架可用于解决一般泛函空间中梯度变小的问题。以具体实例而言，我们获得了标准$\ell_1$设置（在$\ell_{\infty}$范数下的小梯度）下近乎最优的方法，实质上匹配了Nesterov（2012）此前仅适用于欧几里得设置的界限。最终，我们证明了这些复合方法广泛适用于回归及其他以正则化为核心的一类优化问题，所得复杂度界限或为全新结果，或与现有最优结果持平。