This paper studies a new variant of the stochastic multi-armed bandits problem where auxiliary information about the arm rewards is available in the form of control variates. In many applications like queuing and wireless networks, the arm rewards are functions of some exogenous variables. The mean values of these variables are known a priori from historical data and can be used as control variates. Leveraging the theory of control variates, we obtain mean estimates with smaller variance and tighter confidence bounds. We develop an improved upper confidence bound based algorithm named UCB-CV and characterize the regret bounds in terms of the correlation between rewards and control variates when they follow a multivariate normal distribution. We also extend UCB-CV to other distributions using resampling methods like Jackknifing and Splitting. Experiments on synthetic problem instances validate performance guarantees of the proposed algorithms.

翻译：本文研究一种新型的随机多武装匪徒问题,即以控制变异的形式提供关于手臂奖励的辅助信息。在许多应用中,如排队和无线网络,手臂奖励是一些外源变量的函数。这些变量的平均值从历史数据中先验地得知,可以用作控制变异。利用控制变异理论,我们获得平均估计数,但差异较小,信任界限更紧。我们开发了一种以UCB-CV为名的更高级信任约束算法,并用多种变异分布后奖赏和控制变异之间的相关性来描述遗憾界限。我们还利用Jackkfining和分解等重新标注方法将UCB-CV推广到其他分布。在合成问题实例上进行的实验验证了拟议算法的性能保障。