The stochastic gradient descent (SGD) algorithm has been widely used in statistical estimation for large-scale data due to its computational and memory efficiency. While most existing works focus on the convergence of the objective function or the error of the obtained solution, we investigate the problem of statistical inference of true model parameters based on SGD when the population loss function is strongly convex and satisfies certain smoothness conditions. Our main contributions are two-fold. First, in the fixed dimension setup, we propose two consistent estimators of the asymptotic covariance of the average iterate from SGD: (1) a plug-in estimator, and (2) a batch-means estimator, which is computationally more efficient and only uses the iterates from SGD. Both proposed estimators allow us to construct asymptotically exact confidence intervals and hypothesis tests. Second, for high-dimensional linear regression, using a variant of the SGD algorithm, we construct a debiased estimator of each regression coefficient that is asymptotically normal. This gives a one-pass algorithm for computing both the sparse regression coefficients and confidence intervals, which is computationally attractive and applicable to online data.

翻译：随机梯度下降（SGD）算法因其计算和内存效率而被广泛应用于大规模数据的统计估计。尽管现有研究大多关注目标函数的收敛性或所得解的误差，本文研究了在总体损失函数强凸且满足特定光滑性条件下，基于SGD的真实模型参数统计推断问题。我们的主要贡献有两方面：首先，在固定维度设定下，我们提出了两个SGD平均迭代渐近协方差的一致估计量：（1）插件估计量，以及（2）批均值估计量，后者计算效率更高且仅需使用SGD迭代结果。这两个估计量均能构建渐近精确的置信区间和假设检验。其次，针对高维线性回归，通过使用SGD算法的变体，我们构建了每个回归系数的去偏估计量，该估计量具有渐近正态性。这提供了一种单遍算法，可同时计算稀疏回归系数和置信区间，具有计算优势且适用于在线数据。