In this work, we investigate the problem of adapting to the presence or absence of causal structure in multi-armed bandit problems. In addition to the usual reward signal, we assume the learner has access to additional variables, observed in each round after acting. When these variables $d$-separate the action from the reward, existing work in causal bandits demonstrates that one can achieve strictly better (minimax) rates of regret (Lu et al., 2020). Our goal is to adapt to this favorable "conditionally benign" structure, if it is present in the environment, while simultaneously recovering worst-case minimax regret, if it is not. Notably, the learner has no prior knowledge of whether the favorable structure holds. In this paper, we establish the Pareto optimal frontier of adaptive rates. We prove upper and matching lower bounds on the possible trade-offs in the performance of learning in conditionally benign and arbitrary environments, resolving an open question raised by Bilodeau et al. (2022). Furthermore, we are the first to obtain instance-dependent bounds for causal bandits, by reducing the problem to the linear bandit setting. Finally, we examine the common assumption that the marginal distributions of the post-action contexts are known and show that a nontrivial estimate is necessary for better-than-worst-case minimax rates.

翻译：本研究探讨了在多臂赌博机问题中适应因果结构存在与否的问题。除常规奖励信号外，我们假设学习者在每轮行动后能观测到额外变量。当这些变量通过$d$-分离将行动与奖励解耦时，现有因果赌博机研究证明可获得严格更优的（极小极大）遗憾率（Lu等人，2020）。我们的目标是：若环境中存在这种有利的"条件良性"结构则适应之，若不存在则恢复最坏情况的极小极大遗憾。值得注意的是，学习者事先并不知晓该有利结构是否存在。本文建立了自适应速率的帕累托最优前沿，通过证明条件良性环境与任意环境中学习性能权衡的上界及匹配下界，解决了Bilodeau等人（2022）提出的开放性问题。此外，我们通过将问题归约至线性赌博机设定，首次获得了因果赌博机的实例相关界。最后，我们检验了"行动后背景变量的边缘分布已知"这一常见假设，证明要获得优于最坏情况极小极大遗憾的速率，必须进行非平凡估计。