We introduce $k$-lazyGD, an online learning algorithm that bridges the gap between greedy Online Gradient Descent (OGD, for $k=1$) and lazy GD/dual-averaging (for $k=T$), creating a spectrum between reactive and stable updates. We analyze this spectrum in Smoothed Online Convex Optimization (SOCO), where the learner incurs both hitting and movement costs. Our main contribution is establishing that laziness is possible without sacrificing hitting performance: we prove that $k$-lazyGD achieves the optimal dynamic regret $\mathcal{O}(\sqrt{(P_T+1)T})$ for any laziness slack $k$ up to $Θ(\sqrt{T/P_T})$, where $P_T$ is the comparator path length. This result formally connects the allowable laziness to the comparator's shifts, showing that $k$-lazyGD can retain the inherently small movements of lazy methods without compromising tracking ability. We base our analysis on the Follow the Regularized Leader (FTRL) framework, and derive a matching lower bound. Since the slack depends on $P_T$, an ensemble of learners with various slacks is used, yielding a method that is provably stable when it can be, and agile when it must be.

翻译：本文提出$k$-惰性梯度下降算法（$k$-lazyGD），该在线学习算法在贪婪的在线梯度下降（OGD，对应$k=1$）与惰性梯度下降/对偶平均法（对应$k=T$）之间构建了连续谱系，实现了反应式更新与稳定式更新之间的动态平衡。我们在平滑在线凸优化（SOCO）框架下分析该谱系，其中学习者需同时承担命中成本与移动成本。我们的核心贡献在于证明惰性机制可在不牺牲命中性能的前提下实现：对于任意满足$k \leq \Theta(\sqrt{T/P_T})$的惰性松弛参数$k$，我们证明$k$-lazyGD能够达到最优动态遗憾界$\mathcal{O}(\sqrt{(P_T+1)T})$，其中$P_T$为比较器路径长度。该结果形式化地建立了允许惰性程度与比较器变化幅度之间的关联，表明$k$-lazyGD既能保持惰性方法固有的小幅移动特性，又不损失轨迹追踪能力。我们的分析基于跟随正则化领导者（FTRL）框架，并推导出匹配的下界。由于松弛参数依赖于$P_T$，我们采用多松弛参数学习器集成策略，从而构建出在可能时保持稳定、在必要时快速响应的可证明有效方法。