Follow-the-regularized-leader framework has shown effectiveness and flexibility in online learning problems, where the choice of learning rates are known to be crucial. Recently, adaptive learning rates defined in terms of the arm-selection probabilities, obtained by solving convex optimization, have achieved improved best-of-both-worlds (BOBW) guarantees in various bandit problems. In contrast, BOBW guarantees for its computationally efficient alternative, follow-the-perturbed-leader (FTPL), remain relatively limited since its optimization-free nature ironically makes the design of adaptive, probability-dependent learning rates non-trivial. To address this challenge, we propose an adaptive learning rate for FTPL by introducing surrogate probability functions that can be computed only from the available quantities, without requiring the exact probabilities. Based on these learning rates with surrogate functions, we provide the BOBW guarantee for FTPL with Pareto perturbations for any shape parameter $α>1$, generalizing prior results restricted to specific choices of $α=2$. We further show the BOBW guarantees for FTPL with adaptive learning rates in the bandit problem with expert advices. Our approach preserves the computational simplicity of FTPL while enabling probability-dependent adaptivity, and the surrogate-based methodology may be of independent interest in other algorithmic frameworks beyond FTPL and learning rate designs.

翻译：跟随正则化领导者框架在在线学习问题中展现出有效性和灵活性，其中学习率的选择至关重要。近期，通过求解凸优化获得的、基于臂选择概率定义的自适应学习率，已在多种赌博机问题中实现了改进的最佳双世界（BOBW）保证。然而，对于其计算高效替代方案——跟随扰动领导者（FTPL），BOBW保证仍相对有限，因为其无优化的特性反而使得设计依赖于概率的自适应学习率变得困难。为应对这一挑战，我们通过引入只需利用现有量计算、无需精确概率的替代概率函数，为FTPL提出了自适应学习率。基于这些使用替代函数的学习率，我们为采用帕累托扰动且任意形状参数$α>1$的FTPL提供了BOBW保证，将先前仅限于特定选择$α=2$的结果推广至一般情形。我们进一步展示了在带专家建议的赌博机问题中，具有自适应学习率的FTPL的BOBW保证。该方法在保持FTPL计算简洁性的同时实现了依赖概率的自适应能力，且基于替代函数的方法论可能对超越FTPL的其他算法框架及学习率设计具有独立参考价值。