This paper studies the sample average approximation (SAA) in solving convex or strongly convex stochastic programming problems. Under some common regularity conditions, we show -- perhaps for the first time -- that the SAA's sample complexity can be completely free from any quantification of metric entropy (such as the logarithm of the covering number), leading to a significantly more efficient rate with dimensionality $d$ than most existing results. From the newly established complexity bounds, an important revelation is that the SAA and the canonical stochastic mirror descent (SMD) method, two mainstream solution approaches to SP, entail almost identical rates of sample efficiency, rectifying a long-standing theoretical discrepancy of the SAA from the SMD by the order of $O(d)$. Furthermore, this paper explores non-Lipschitzian scenarios where the SAA maintains provable efficacy, whereas corresponding results for the SMD remain unexplored, indicating the potential of the SAA's better applicability in some irregular settings.
翻译:本文研究求解凸或强凸随机规划问题的样本均值近似(SAA)方法。在若干常见正则性条件下,我们首次证明SAA的样本复杂度可完全摆脱任何度量熵的量化指标(如覆盖数的对数),从而在维度d上获得比现有结果显著更高的效率。从新建立的复杂度界中,一个重要发现是:作为SP领域两种主流求解途径的SAA与经典随机镜像下降(SMD)方法,在样本效率方面具有几乎相同的收敛速率,这纠正了长期存在的SAA相对SMD存在O(d)阶理论偏差的问题。此外,本文探讨了非Lipschitz情景下SAA仍保持可证明有效性的现象,而SMD的对应结果尚待研究,这表明SAA在某些非正则场景中具有更好的适用潜力。