Adaptivity to the difficulties of a problem is a key property in sequential decision-making problems to broaden the applicability of algorithms. Follow-the-Regularized-Leader (FTRL) has recently emerged as one of the most promising approaches for obtaining various types of adaptivity in bandit problems. Aiming to further generalize this adaptivity, we develop a generic adaptive learning rate, called Stability-Penalty-Adaptive (SPA) learning rate for FTRL. This learning rate yields a regret bound jointly depending on stability and penalty of the algorithm, into which the regret of FTRL is typically decomposed. With this result, we establish several algorithms with three types of adaptivity: sparsity, game-dependency, and Best-of-Both-Worlds (BOBW). Sparsity frequently appears in real-world problems. However, existing sparse multi-armed bandit algorithms with $k$-arms assume that the sparsity level $s \leq k$ is known in advance, which is often not the case in real-world scenarios. To address this problem, with the help of the new learning rate framework, we establish $s$-agnostic algorithms with regret bounds of $\tilde{O}(\sqrt{sT})$ in the adversarial regime for $T$ rounds, which matches the existing lower bound up to a logarithmic factor. Meanwhile, BOBW algorithms aim to achieve a near-optimal regret in both the stochastic and adversarial regimes. Leveraging the new adaptive learning rate framework and a novel analysis to bound the variation in FTRL output in response to changes in a regularizer, we establish the first BOBW algorithm with a sparsity-dependent bound. Additionally, we explore partial monitoring and demonstrate that the proposed learning rate framework allows us to achieve a game-dependent bound and the BOBW simultaneously.

翻译：对问题难度的适应性是序贯决策问题中扩展算法适用性的关键属性。近年来，跟随正则化领导者（FTRL）已成为赌臂问题中实现多种适应性的最有前景方法之一。为进一步推广这种适应性，我们为FTRL开发了一种通用自适应学习率，称为稳定性-惩罚自适应（SPA）学习率。该学习率产生的遗憾界同时依赖于算法的稳定性与惩罚项——这正是FTRL遗憾通常分解的两个组成部分。基于这一结果，我们建立了几种具备三类适应性的算法：稀疏性、博弈依赖性和两全其美（BOBW）。稀疏性在实际问题中频繁出现，然而现有基于$k$臂的稀疏多臂赌臂算法假设稀疏度水平$s \leq k$事先已知，这在现实场景中往往不成立。为解决该问题，借助新的学习率框架，我们建立了$s$未知的算法，其在对抗性环境下$T$轮博弈的遗憾界为$\tilde{O}(\sqrt{sT})$，与现有下界仅差对数因子。与此同时，BOBW算法旨在随机环境与对抗性环境下均达到近最优遗憾。通过结合新的自适应学习率框架与一种分析正则化器变化下FTRL输出变化界限的新方法，我们建立了首个具有稀疏性依赖界的BOBW算法。此外，我们探索了部分监控问题，并证明所提出的学习率框架能够同时实现博弈依赖性界与BOBW。