This paper develops a new dimension-free Azuma-Hoeffding type bound on summation norm of a martingale difference sequence with random individual bounds. With this novel result, we provide high-probability bounds for the gradient norm estimator in the proposed algorithm Prob-SARAH, which is a modified version of the StochAstic Recursive grAdient algoritHm (SARAH), a state-of-art variance reduced algorithm that achieves optimal computational complexity in expectation for the finite sum problem. The in-probability complexity by Prob-SARAH matches the best in-expectation result up to logarithmic factors. Empirical experiments demonstrate the superior probabilistic performance of Prob-SARAH on real datasets compared to other popular algorithms.

翻译：本文发展了一种新的无维Azuma-Hoeffding类型界，用于具有随机个体界的鞅差序列的求和范数。基于这一新颖结果，我们为所提出的算法Prob-SARAH中的梯度范数估计量提供了高概率界。Prob-SARAH是随机递归梯度算法（SARAH）的改进版本，后者是一种针对有限和问题在期望意义上达到最优计算复杂度的最先进方差缩减算法。Prob-SARAH的概率复杂度与最优期望结果相差仅对数因子。真实数据集上的实证实验表明，与其他流行算法相比，Prob-SARAH具有优越的概率性能。