A novel approach is given to overcome the computational challenges of the full-matrix Adaptive Gradient algorithm (Full AdaGrad) in stochastic optimization. By developing a recursive method that estimates the inverse of the square root of the covariance of the gradient, alongside a streaming variant for parameter updates, the study offers efficient and practical algorithms for large-scale applications. This innovative strategy significantly reduces the complexity and resource demands typically associated with full-matrix methods, enabling more effective optimization processes. Moreover, the convergence rates of the proposed estimators and their asymptotic efficiency are given. Their effectiveness is demonstrated through numerical studies.

翻译：本文提出了一种新方法，以克服随机优化中全矩阵自适应梯度算法（Full AdaGrad）的计算挑战。通过开发一种递归方法来估计梯度协方差平方根的逆，并结合用于参数更新的流式变体，本研究为大规模应用提供了高效且实用的算法。这一创新策略显著降低了通常与全矩阵方法相关的复杂性和资源需求，从而实现了更有效的优化过程。此外，本文给出了所提出估计器的收敛速率及其渐近效率，并通过数值研究证明了其有效性。