Optimization problems arising in data science have given rise to a number of new derivative-based optimization methods. Such methods often use standard smoothness assumptions -- namely, global Lipschitz continuity of the gradient function -- to establish a convergence theory. Unfortunately, in this work, we show that common optimization problems from data science applications are not globally Lipschitz smooth, nor do they satisfy some more recently developed smoothness conditions in literature. Instead, we show that such optimization problems are better modeled as having locally Lipschitz continuous gradients. We then construct explicit examples satisfying this assumption on which existing classes of optimization methods are either unreliable or experience an explosion in evaluation complexity. In summary, we show that optimization problems arising in data science are particularly difficult to solve, and that there is a need for methods that can reliably and practically solve these problems.

翻译：数据科学中的优化问题催生了多种基于导数的优化新方法。这些方法通常借助标准光滑性假设（即梯度函数的全局Lipschitz连续性）来建立收敛理论。然而，本研究表明，数据科学应用中的常见优化问题既不具备全局Lipschitz光滑性，也不满足文献中最近提出的某些光滑性条件。相反，我们证明此类优化问题更适合建模为具有局部Lipschitz连续梯度的模型。随后，我们构造了满足这一假设的明确示例，现有优化方法在该示例上或不可靠，或评估复杂度出现爆炸性增长。综上，我们揭示了数据科学中的优化问题求解难度极大，亟需能够可靠且高效解决此类问题的方法。