In this paper, we propose an online convex optimization method with two different levels of adaptivity. On a higher level, our method is agnostic to the specific type and curvature of the loss functions, while at a lower level, it can exploit the niceness of the environments and attain problem-dependent guarantees. To be specific, we obtain $\mathcal{O}(\ln V_T)$, $\mathcal{O}(d \ln V_T)$ and $\hat{\mathcal{O}}(\sqrt{V_T})$ regret bounds for strongly convex, exp-concave and convex loss functions, respectively, where $d$ is the dimension, $V_T$ denotes problem-dependent gradient variations and $\hat{\mathcal{O}}(\cdot)$-notation omits logarithmic factors on $V_T$. Our result finds broad implications and applications. It not only safeguards the worst-case guarantees, but also implies the small-loss bounds in analysis directly. Besides, it draws deep connections with adversarial/stochastic convex optimization and game theory, further validating its practical potential. Our method is based on a multi-layer online ensemble incorporating novel ingredients, including carefully-designed optimism for unifying diverse function types and cascaded corrections for algorithmic stability. Remarkably, despite its multi-layer structure, our algorithm necessitates only one gradient query per round, making it favorable when the gradient evaluation is time-consuming. This is facilitated by a novel regret decomposition equipped with customized surrogate losses.

翻译：本文提出了一种具有两种不同自适应级别的在线凸优化方法。在较高层面，我们的方法不依赖于损失函数的具体类型与曲率；而在较低层面，该方法能够利用环境的优良性质，实现问题相关的保证。具体而言，对于强凸、指数凹和凸损失函数，我们分别获得了$\mathcal{O}(\ln V_T)$、$\mathcal{O}(d \ln V_T)$和$\hat{\mathcal{O}}(\sqrt{V_T})$的遗憾界，其中$d$为数据维度，$V_T$表示问题相关的梯度变化量，而$\hat{\mathcal{O}}(\cdot)$符号省略了$V_T$的对数因子。我们的结果具有广泛的意义和应用价值：它不仅确保了最坏情形下的性能保证，还能直接推导出分析中的小损失界。此外，该结果与对抗/随机凸优化及博弈论建立了深层联系，进一步验证了其实际应用的潜力。该方法基于一种多层在线集成框架，并融入了多项创新设计，包括为统一不同函数类型而精心构造的乐观估计，以及用于保证算法稳定性的级联修正项。值得关注的是，尽管算法具有多层结构，每轮只需一次梯度查询，这使得它在梯度评估耗时的情况下具有显著优势。这一特性得益于一种创新性的遗憾分解方法，该方法配合定制化的替代损失函数实现。