Momentum methods, such as Polyak's Heavy Ball, are the standard for training deep networks but suffer from curvature-induced bias in stochastic settings, limiting convergence to suboptimal $\mathcal{O}(ε^{-4})$ rates. Existing corrections typically require expensive auxiliary sampling or restrictive smoothness assumptions. We propose \textbf{RanSOM}, a unified framework that eliminates this bias by replacing deterministic step sizes with randomized steps drawn from distributions with mean $η_t$. This modification allows us to leverage Stein-type identities to compute an exact, unbiased estimate of the momentum bias using a single Hessian-vector product computed jointly with the gradient, avoiding auxiliary queries. We instantiate this framework in two algorithms: \textbf{RanSOM-E} for unconstrained optimization (using exponentially distributed steps) and \textbf{RanSOM-B} for constrained optimization (using beta-distributed steps to strictly preserve feasibility). Theoretical analysis confirms that RanSOM recovers the optimal $\mathcal{O}(ε^{-3})$ convergence rate under standard bounded noise, and achieves optimal rates for heavy-tailed noise settings ($p \in (1, 2]$) without requiring gradient clipping.

翻译：动量方法（如Polyak重球法）是训练深度网络的标准技术，但在随机场景中会因曲率诱导偏差而受限，收敛速率仅能达到次优的$\mathcal{O}(ε^{-4})$。现有校正方法通常需要昂贵的辅助采样或严格的平滑性假设。我们提出\textbf{RanSOM}——一个通过将确定性步长替换为从均值为$η_t$的分布中采样的随机步长来消除该偏差的统一框架。此修改使我们能够利用Stein型恒等式，通过单个与梯度联合计算的海森-向量积来精确无偏地估计动量偏差，从而避免辅助查询。我们将该框架具体实现为两种算法：用于无约束优化的\textbf{RanSOM-E}（采用指数分布步长）和用于约束优化的\textbf{RanSOM-B}（采用贝塔分布步长以严格保持可行性）。理论分析证实，在标准有界噪声条件下RanSOM可恢复最优的$\mathcal{O}(ε^{-3})$收敛速率，并在无需梯度裁剪的情况下对重尾噪声场景（$p \in (1, 2]$）实现最优收敛速率。