Wasserstein distributionally robust optimization (\textsf{WDRO}) is a popular model to enhance the robustness of machine learning with ambiguous data. However, the complexity of \textsf{WDRO} can be prohibitive in practice since solving its ``minimax'' formulation requires a great amount of computation. Recently, several fast \textsf{WDRO} training algorithms for some specific machine learning tasks (e.g., logistic regression) have been developed. However, the research on designing efficient algorithms for general large-scale \textsf{WDRO}s is still quite limited, to the best of our knowledge. \textit{Coreset} is an important tool for compressing large dataset, and thus it has been widely applied to reduce the computational complexities for many optimization problems. In this paper, we introduce a unified framework to construct the $\epsilon$-coreset for the general \textsf{WDRO} problems. Though it is challenging to obtain a conventional coreset for \textsf{WDRO} due to the uncertainty issue of ambiguous data, we show that we can compute a ``dual coreset'' by using the strong duality property of \textsf{WDRO}. Also, the error introduced by the dual coreset can be theoretically guaranteed for the original \textsf{WDRO} objective. To construct the dual coreset, we propose a novel grid sampling approach that is particularly suitable for the dual formulation of \textsf{WDRO}. Finally, we implement our coreset approach and illustrate its effectiveness for several \textsf{WDRO} problems in the experiments.
翻译:Wasserstein分布鲁棒优化(\textsf{WDRO})是一种流行的模型,用于增强机器学习在处理模糊数据时的鲁棒性。然而,在实际应用中,\textsf{WDRO}的复杂性可能令人望而却步,因为求解其“极小化-极大化”公式需要大量计算。近年来,针对某些特定机器学习任务(例如逻辑回归)开发了几种快速的\textsf{WDRO}训练算法。但据我们所知,关于设计通用大规模\textsf{WDRO}高效算法的研究仍然相当有限。核心集(Coreset)是压缩大数据集的重要工具,因此已被广泛应用于降低许多优化问题的计算复杂度。在本文中,我们引入了一个统一框架,用于构建通用\textsf{WDRO}问题的$\epsilon$-核心集。尽管由于模糊数据的不确定性问题,获得\textsf{WDRO}的传统核心集颇具挑战性,但我们表明,可以利用\textsf{WDRO}的强对偶性来计算“对偶核心集”。此外,对偶核心集引入的误差可以在理论上保证相对于原始\textsf{WDRO}目标函数的有效性。为了构建对偶核心集,我们提出了一种新颖的网格采样方法,该方法特别适用于\textsf{WDRO}的对偶公式。最后,我们实现了核心集方法,并通过多个\textsf{WDRO}问题的实验展示了其有效性。