In this paper, we provide novel tail bounds on the optimization error of Stochastic Mirror Descent for convex and Lipschitz objectives. Our analysis extends the existing tail bounds from the classical light-tailed Sub-Gaussian noise case to heavier-tailed noise regimes. We study the optimization error of the last iterate as well as the average of the iterates. We instantiate our results in two important cases: a class of noise with exponential tails and one with polynomial tails. A remarkable feature of our results is that they do not require an upper bound on the diameter of the domain. Finally, we support our theory with illustrative experiments that compare the behavior of the average of the iterates with that of the last iterate in heavy-tailed noise regimes.

翻译：本文提供了针对凸且Lipschitz目标的随机镜像下降优化误差的新型尾界。我们的分析将经典轻尾亚高斯噪声情形下的现有尾界推广至重尾噪声区域。我们研究了末次迭代以及迭代平均值的优化误差，并在两种重要情形下实例化结果：一类具有指数尾部的噪声和一类具有多项式尾部的噪声。值得注意的是，我们的结果无需对定义域的直径施加上界。最后，我们通过对比重尾噪声区域中迭代平均值与末次迭代行为的实验，为理论提供了支撑。