We leverage offline data to facilitate online learning in stochastic multi-armed bandits. The probability distributions that govern the offline data and the online rewards can be different. Without any non-trivial upper bound on their difference, we show that no non-anticipatory policy can outperform the UCB policy by (Auer et al. 2002), even in the presence of offline data. In complement, we propose an online policy MIN-UCB, which outperforms UCB when a non-trivial upper bound is given. MIN-UCB adaptively chooses to utilize the offline data when they are deemed informative, and to ignore them otherwise. MIN-UCB is shown to be tight in terms of both instance independent and dependent regret bounds. Finally, we corroborate the theoretical results with numerical experiments.

翻译：我们利用离线数据促进随机多臂老虎机中的在线学习。控制离线数据与在线奖励的概率分布可能不同。在不假设两者差异存在任何非平凡上界的情况下，我们证明即使存在离线数据，任何非预期策略都无法超越Auer等人（2002）提出的UCB策略。作为补充，我们提出在线策略MIN-UCB，当给定非平凡上界时，该策略表现优于UCB。MIN-UCB自适应地选择在离线数据被认为具有信息性时加以利用，否则忽略它们。研究表明，MIN-UCB在实例无关和实例相关的遗憾界方面均达到紧致性。最后，我们通过数值实验验证了理论结果。