Reliable decision-making with streaming data requires principled uncertainty quantification of online methods. While first-order methods enable efficient iterate updates, their inference procedures still require updating proper (covariance) matrices, incurring $O(d^2)$ time and memory complexity, and are sensitive to ill-conditioning and noise heterogeneity of the problem. This costly inference task offers an opportunity for more robust second-order methods, which are, however, bottlenecked by solving Newton systems with $O(d^3)$ complexity. In this paper, we address this gap by studying an online Newton method with Hessian averaging, where the Newton direction at each step is approximately computed using a sketch-and-project solver with Nesterov's acceleration, matching $O(d^2)$ complexity of first-order methods. For the proposed method, we quantify its uncertainty arising from both random data and randomized computation. Under standard smoothness and moment conditions, we establish global almost-sure convergence, prove asymptotic normality of the last iterate with a limiting covariance characterized by a Lyapunov equation, and develop a fully online covariance estimator with non-asymptotic convergence guarantees. We also connect the resulting uncertainty quantification to that of exact and sketched Newton methods without Nesterov's acceleration. Extensive experiments on regression models demonstrate the superiority of the proposed method for online inference.

翻译：可靠地处理流式数据并做出决策，需要对在线方法进行严谨的不确定性量化。虽然一阶方法能够实现高效的迭代更新，但其推断过程仍需更新适当（协方差）矩阵，带来$O(d^2)$的时间和内存复杂度，并且对问题的病态性和噪声异质性敏感。这一高成本的推断任务为更鲁棒的二阶方法提供了机会，然而二阶方法受限于求解$O(d^3)$复杂度的牛顿系统。在本文中，我们通过研究一种带有海森矩阵平均的在线牛顿方法来弥补这一差距，其中每一步的牛顿方向使用带有涅斯捷罗夫加速的素描-投影求解器近似计算，匹配一阶方法的$O(d^2)$复杂度。对于所提出的方法，我们量化了由随机数据和随机计算共同引起的不确定性。在标准平滑性和矩条件下，我们建立了全局几乎必然收敛性，证明了最后迭代的渐近正态性，其中极限协方差由李雅普诺夫方程刻画，并开发了一个完全在线且具有非渐近收敛保证的协方差估计器。我们还将由此产生的不确定性量化与无涅斯捷罗夫加速的精确和素描牛顿方法进行了关联。在回归模型上的广泛实验证明了所提方法在在线推断中的优越性。