We develop a framework for the average-case analysis of random quadratic problems and derive algorithms that are optimal under this analysis. This yields a new class of methods that achieve acceleration given a model of the Hessian's eigenvalue distribution. We develop explicit algorithms for the uniform, Marchenko-Pastur, and exponential distributions. These methods are momentum-based algorithms, whose hyper-parameters can be estimated without knowledge of the Hessian's smallest singular value, in contrast with classical accelerated methods like Nesterov acceleration and Polyak momentum. Through empirical benchmarks on quadratic and logistic regression problems, we identify regimes in which the the proposed methods improve over classical (worst-case) accelerated methods.

翻译：我们建立了一个随机二次问题平均情况分析框架，并推导出在该分析下最优的算法。这产生了一类新方法，在已知Hessian矩阵特征值分布模型的情况下能够实现加速。我们针对均匀分布、Marchenko-Pastur分布和指数分布开发了显式算法。这些方法基于动量机制，与Nesterov加速和Polyak动量等经典（最坏情况）加速方法不同，其超参数无需知晓Hessian矩阵的最小奇异值即可估计。通过二次回归和逻辑回归问题的经验基准测试，我们识别了所提出方法相较于经典（最坏情况）加速方法有所改进的领域。