We present a family of generalized Hessian estimators of the objective using random direction stochastic approximation (RDSA) by utilizing only noisy function measurements. The form of each estimator and the order of the bias depend on the number of function measurements. In particular, we demonstrate that estimators with more function measurements exhibit lower-order estimation bias. We show the asymptotic unbiasedness of the estimators. We also perform asymptotic and non-asymptotic convergence analyses for stochastic Newton methods that incorporate our generalized Hessian estimators. Finally, we perform numerical experiments to validate our theoretical findings.

翻译：本文提出了一族基于随机方向随机逼近（RDSA）的广义Hessian矩阵估计器，该估计器仅需利用含噪声的函数测量值。每个估计器的具体形式及其偏差阶数取决于所使用的函数测量次数。特别地，我们证明了采用更多函数测量值的估计器具有更低阶的估计偏差。我们验证了这些估计器的渐近无偏性，并对结合了广义Hessian估计器的随机牛顿方法进行了渐近与非渐近收敛性分析。最后，我们通过数值实验验证了理论结论。