A novel approach is given to overcome the computational challenges of the full-matrix Adaptive Gradient algorithm (Full AdaGrad) in stochastic optimization. By developing a recursive method that estimates the inverse of the square root of the covariance of the gradient, alongside a streaming variant for parameter updates, the study offers efficient and practical algorithms for large-scale applications. This innovative strategy significantly reduces the complexity and resource demands typically associated with full-matrix methods, enabling more effective optimization processes. Moreover, the convergence rates of the proposed estimators and their asymptotic efficiency are given. Their effectiveness is demonstrated through numerical studies.

翻译：针对随机优化中全矩阵自适应梯度算法（Full AdaGrad）存在的计算挑战，本文提出一种新型解决方法。通过开发递归方法估计梯度协方差平方根的逆矩阵，并结合参数更新的流式变体，本研究为大规模应用提供了高效实用的算法。该创新策略显著降低了全矩阵方法通常所需的计算复杂度和资源需求，使得优化过程更加高效。此外，本文给出了所提出估计量的收敛速率及其渐近效率，并通过数值研究验证了其有效性。