We provide abstract, general and highly uniform rates of asymptotic regularity for a generalized stochastic Halpern-style iteration, which incorporates a second mapping in the style of a Krasnoselskii-Mann iteration. This iteration is general in two ways: First, it incorporates stochasticity completely abstractly, rather than fixing a sampling method; second, it includes as special cases stochastic versions of various schemes from the optimization literature, including Halpern's iteration as well as a Krasnoselskii-Mann iteration with Tikhonov regularization terms in the sense of Boţ, Csetnek and Meier (where this stochastic variant of the latter is considered for the first time in this paper). For these specific cases, we obtain linear rates of asymptotic regularity, matching (or improving) the currently best known rates for these iterations in stochastic optimization, and quadratic rates of asymptotic regularity are obtained in the context of inner product spaces for the general iteration. We conclude by discussing how variance can be managed in practice through sampling methods in the style of minibatching, how our convergence rates can be adapted to provide oracle complexity bounds, and by sketching how the schemes presented here can be instantiated in the context of reinforcement learning to yield novel methods for Q-learning.

翻译：我们针对一类广义随机Halpern型迭代，提供抽象、通用且高度一致的渐近正则性收敛速率，该迭代以Krasnoselskii-Mann迭代的风格引入了第二个映射。该迭代在两方面具有通用性：首先，它以完全抽象的方式纳入随机性，而非固定采样方法；其次，其特例涵盖了优化文献中的多种随机化方案，包括Halpern迭代以及带有Boţ、Csetnek和Meier提出的Tikhonov正则化项的Krasnoselskii-Mann迭代（其中后者的随机变体在本文中首次被考虑）。针对这些具体情形，我们获得了渐近正则性的线性收敛速率，与随机优化中当前已知最优速率相匹配（或有所改进）；在内积空间背景下，该广义迭代可获得渐近正则性的二次收敛速率。最后，我们讨论了如何通过小批量采样方法在实践中控制方差，如何调整收敛速率以提供预言机复杂度界，并概述了如何将本文提出的方案实例化于强化学习场景，从而为Q学习衍生出新型方法。