We propose a new online algorithm for cumulative regret minimization in a stochastic linear bandit. The algorithm pulls the arm with the highest estimated reward in a linear model trained on its perturbed history. Therefore, we call it perturbed-history exploration in a linear bandit (LinPHE). The perturbed history is a mixture of observed rewards and randomly generated i.i.d. pseudo-rewards. We derive a $\tilde{O}(d \sqrt{n})$ gap-free bound on the $n$-round regret of LinPHE, where $d$ is the number of features. The key steps in our analysis are new concentration and anti-concentration bounds on the weighted sum of Bernoulli random variables. To show the generality of our design, we generalize LinPHE to a logistic model. We evaluate our algorithms empirically and show that they are practical.

翻译：本文提出一种新的在线算法，用于随机线性Bandit问题中累积遗憾的最小化。该算法基于在扰动历史数据上训练的线性模型，选择具有最高估计奖励的臂。因此，我们将其称为线性Bandit中的扰动历史探索（LinPHE）。扰动历史由观测到的奖励与随机生成的独立同分布伪奖励混合构成。我们推导出LinPHE在$n$轮遗憾上的$\tilde{O}(d \sqrt{n})$阶无间隙界，其中$d$为特征维数。分析的关键步骤是对伯努利随机变量加权和建立新的集中与反集中界。为展示设计的普适性，我们将LinPHE推广至逻辑斯蒂模型。通过实验评估，我们证明所提算法具有实际应用价值。