In this paper, we propose practical normalized stochastic first-order methods with Polyak momentum, multi-extrapolated momentum, and recursive momentum for solving unconstrained optimization problems. These methods employ dynamically updated algorithmic parameters and do not require explicit knowledge of problem-dependent quantities such as the Lipschitz constant or noise bound. We establish first-order oracle complexity results for finding approximate stochastic stationary points under heavy-tailed noise and weakly average smoothness conditions -- both of which are weaker than the commonly used bounded variance and mean-squared smoothness assumptions. Our complexity bounds either improve upon or match the best-known results in the literature. Numerical experiments are presented to demonstrate the practical effectiveness of the proposed methods.

翻译：本文提出了实用的归一化随机一阶方法，结合Polyak动量、多重外推动量和递归动量，用于求解无约束优化问题。这些方法采用动态更新的算法参数，无需显式了解问题相关的量（如Lipschitz常数或噪声界）。在重尾噪声和弱平均光滑性条件下——这两者均弱于常用的有界方差和均方光滑性假设——我们建立了寻找近似随机稳定点的一阶预言机复杂度结果。我们的复杂度界限改进或匹配了文献中已知的最佳结果。数值实验展示了所提方法的实际有效性。