In this paper, we establish the non-asymptotic validity of the multiplier bootstrap procedure for constructing the confidence sets using the Stochastic Gradient Descent (SGD) algorithm. Under appropriate regularity conditions, our approach avoids the need to approximate the limiting covariance of Polyak-Ruppert SGD iterates, which allows us to derive approximation rates in convex distance of order up to $1/\sqrt{n}$. Notably, this rate can be faster than the one that can be proven in the Polyak-Juditsky central limit theorem. To our knowledge, this provides the first fully non-asymptotic bound on the accuracy of bootstrap approximations in SGD algorithms. Our analysis builds on the Gaussian approximation results for nonlinear statistics of independent random variables.

翻译：本文建立乘子自助法在使用随机梯度下降（SGD）算法构建置信集时的非渐近有效性。在适当的正则性条件下，我们的方法无需近似Polyak-Ruppert SGD迭代的极限协方差，从而能够在凸距离下推导出阶数高达$1/\sqrt{n}$的逼近速率。值得注意，该速率可能快于Polyak-Juditsky中心极限定理可证明的速率。据我们所知，这是首个关于SGD算法中自助法逼近精度的完全非渐近界。我们的分析建立在独立随机变量非线性统计量高斯逼近结果的基础之上。