We develop a new method of online inference for a vector of parameters estimated by the Polyak-Ruppert averaging procedure of stochastic gradient descent (SGD) algorithms. We leverage insights from time series regression in econometrics and construct asymptotically pivotal statistics via random scaling. Our approach is fully operational with online data and is rigorously underpinned by a functional central limit theorem. Our proposed inference method has a couple of key advantages over the existing methods. First, the test statistic is computed in an online fashion with only SGD iterates and the critical values can be obtained without any resampling methods, thereby allowing for efficient implementation suitable for massive online data. Second, there is no need to estimate the asymptotic variance and our inference method is shown to be robust to changes in the tuning parameters for SGD algorithms in simulation experiments with synthetic data.

翻译：我们开发了一种由Polyak-Ruppert 平均随机梯度下降算法(SGD)算法程序估计的参数矢量的在线推断新方法。 我们利用计量经济学时间序列回归的洞察力,通过随机比例构建无症状关键统计数据。 我们的方法完全使用在线数据,并严格以功能中心限制理论为支撑。 我们提议的推断方法比现有方法具有若干关键优势。 首先,测试统计是以在线方式计算,只有 SGD 代数,而且关键值可以在不使用任何恢复方法的情况下获得,从而能够有效地实施适合于大规模在线数据。 其次,没有必要估计无症状差异,而且我们的推断方法显示对合成数据模拟实验中 SGD 算法调试参数的变化非常有力。