The Rising Multi-Armed Bandit (RMAB) framework models environments where expected rewards of arms increase with plays, which models practical scenarios where performance of each option improves with the repeated usage, such as in robotics and hyperparameter tuning. For instance, in hyperparameter tuning, the validation accuracy of a model configuration (arm) typically increases with each training epoch. A defining characteristic of RMAB is em horizon-dependent optimality: unlike standard settings, the optimal strategy here shifts dramatically depending on the available budget $T$. This implies that knowledge of $T$ yields significantly greater utility in RMAB, empowering the learner to align its decision-making with this shifting optimality. However, the horizon-aware setting remains underexplored. To address this, we propose a novel CUmulative Reward Estimation UCB (CURE-UCB) that explicitly integrates the horizon. We provide a rigorous analysis establishing a new regret upper bound and prove that our method strictly outperforms horizon-agnostic strategies in structured environments like ``linear-then-flat'' instances. Extensive experiments demonstrate its significant superiority over baselines.

翻译：上升多臂赌博机（RMAB）框架对环境中各臂的期望奖励随拉动次数增加而上升的情形进行建模，这模拟了实际场景中每个选项的性能随着重复使用而提高的情况，例如在机器人学和超参数调优中。例如，在超参数调优中，一个模型配置（臂）的验证准确率通常随着每个训练轮次而增加。RMAB的一个决定性特征是时域依赖的最优性：与标准设置不同，此处的最优策略会随着可用预算$T$的不同而发生剧烈变化。这意味着对$T$的认知在RMAB中能带来显著更大的效用，使学习者能够将其决策与这种变化的最优性对齐。然而，时域感知的设置仍未得到充分探索。为解决此问题，我们提出了一种新颖的累积奖励估计UCB（CURE-UCB）方法，该方法明确地整合了时域信息。我们提供了严格的分析，建立了一个新的遗憾上界，并证明了在诸如“线性后平坦”实例等结构化环境中，我们的方法严格优于时域无关策略。大量实验证明了其相对于基线方法的显著优越性。