We study the online overlapping batch-means covariance estimator for Stochastic Gradient Descent (SGD) under Markovian sampling. We show that the convergence rates of the covariance estimator are $O\big(\sqrt{d}\,n^{-1/8}(\log n)^{1/4}\big)$ and $O\big(\sqrt{d}\,n^{-1/8}\big)$ under state-dependent and state-independent Markovian sampling, respectively, with $d$ representing dimensionality and $n$ denoting the number of observations or SGD iterations. Remarkably, these rates match the best-known convergence rate previously established for the independent and identically distributed ($\iid$) case by \cite{zhu2021online}, up to logarithmic factors. Our analysis overcomes significant challenges that arise due to Markovian sampling, leading to the introduction of additional error terms and complex dependencies between the blocks of the batch-means covariance estimator. Moreover, we establish the convergence rate for the first four moments of the $\ell_2$ norm of the error of SGD dynamics under state-dependent Markovian data, which holds potential interest as an independent result. To validate our theoretical findings, we provide numerical illustrations to derive confidence intervals for SGD when training linear and logistic regression models under Markovian sampling. Additionally, we apply our approach to tackle the intriguing problem of strategic classification with logistic regression, where adversaries can adaptively modify features during the training process to increase their chances of being classified in a specific target class.

翻译：我们研究马尔可夫采样下随机梯度下降（SGD）的在线重叠批次均值协方差估计量。我们证明，在状态依赖和状态独立马尔可夫采样下，该协方差估计量的收敛速度分别为$O\big(\sqrt{d}\,n^{-1/8}(\log n)^{1/4}\big)$和$O\big(\sqrt{d}\,n^{-1/8}\big)$，其中$d$表示维度，$n$表示观测值或SGD迭代次数。值得注意的是，这些速度与先前由\cite{zhu2021online}在独立同分布（$\iid$）情况下建立的最佳已知收敛速度相匹配（仅差对数因子）。我们的分析克服了马尔可夫采样带来的重大挑战，这些挑战导致了额外误差项的出现以及批次均值协方差估计量区块之间的复杂依赖关系。此外，我们建立了状态依赖马尔可夫数据下SGD动力学误差$\ell_2$范数的前四阶矩的收敛速度，这作为独立结果具有潜在研究价值。为验证理论发现，我们通过数值实验展示了马尔可夫采样下训练线性回归和逻辑回归模型时SGD的置信区间推导。最后，我们将所提方法应用于逻辑回归中的策略分类这一有趣问题，其中对手可在训练过程中自适应地修改特征以增加被分类到特定目标类别的可能性。