Distributionally Robust Optimization (DRO), as a popular method to train robust models against distribution shift between training and test sets, has received tremendous attention in recent years. In this paper, we propose and analyze stochastic algorithms that apply to both non-convex and convex losses for solving Kullback Leibler divergence constrained DRO problem. Compared with existing methods solving this problem, our stochastic algorithms not only enjoy competitive if not better complexity independent of sample size but also just require a constant batch size at every iteration, which is more practical for broad applications. We establish a nearly optimal complexity bound for finding an $\epsilon$ stationary solution for non-convex losses and an optimal complexity for finding an $\epsilon$ optimal solution for convex losses. Empirical studies demonstrate the effectiveness of the proposed algorithms for solving non-convex and convex constrained DRO problems.

翻译：分布鲁棒优化（DRO）作为一种流行的训练鲁棒模型以应对训练集与测试集分布偏移的方法，近年来受到广泛关注。本文针对求解库尔贝克-莱布勒散度约束DRO问题，提出并分析了适用于非凸损失与凸损失的随机算法。与现有求解该问题的方法相比，本文提出的随机算法不仅具有与样本量无关的竞争性（乃至更优）复杂度，而且每轮迭代仅需恒定批量大小，这使得该方法在实际应用中更具通用性。我们为非凸损失问题建立了近乎最优的$\epsilon$驻点求解复杂度界，并为凸损失问题建立了最优的$\epsilon$最优解求解复杂度界。实验研究验证了所提算法在求解非凸与凸约束DRO问题中的有效性。