We propose a method that achieves near-optimal rates for smooth stochastic convex optimization and requires essentially no prior knowledge of problem parameters. This improves on prior work which requires knowing at least the initial distance to optimality d0. Our method, U-DoG, combines UniXGrad (Kavis et al., 2019) and DoG (Ivgi et al., 2023) with novel iterate stabilization techniques. It requires only loose bounds on d0 and the noise magnitude, provides high probability guarantees under sub-Gaussian noise, and is also near-optimal in the non-smooth case. Our experiments show consistent, strong performance on convex problems and mixed results on neural network training.

翻译：我们提出了一种方法，该方法在光滑随机凸优化问题上实现了近乎最优的收敛速率，且本质上无需预先了解问题参数。这改进了先前需要至少知道初始解距最优解距离d0的工作。我们的方法U-DoG将UniXGrad（Kavis等人，2019）和DoG（Ivgi等人，2023）与新颖的迭代稳定技术相结合。它仅需对d0和噪声幅度的宽松界限，在次高斯噪声下提供高概率保证，并且在非光滑情况下也近乎最优。我们的实验表明，该方法在凸优化问题上表现出一致且强劲的性能，而在神经网络训练上则结果不一。