We introduce exploration via linear loss perturbations (EVILL), a randomised exploration method for structured stochastic bandit problems that works by solving for the minimiser of a linearly perturbed regularised negative log-likelihood function. We show that, for the case of generalised linear bandits, EVILL reduces to perturbed history exploration (PHE), a method where exploration is done by training on randomly perturbed rewards. In doing so, we provide a simple and clean explanation of when and why random reward perturbations give rise to good bandit algorithms. We propose data-dependent perturbations not present in previous PHE-type methods that allow EVILL to match the performance of Thompson-sampling-style parameter-perturbation methods, both in theory and in practice. Moreover, we show an example outside generalised linear bandits where PHE leads to inconsistent estimates, and thus linear regret, while EVILL remains performant. Like PHE, EVILL can be implemented in just a few lines of code.

翻译：我们提出了通过线性损失扰动进行探索的方法（EVILL），这是一种适用于结构化随机赌博机问题的随机化探索方法，其原理是求解线性扰动正则化负对数似然函数的最小化器。我们证明，对于广义线性赌博机的情况，EVILL可简化为扰动历史探索（PHE）——一种通过对随机扰动奖励进行训练来实现探索的方法。在此过程中，我们清晰简洁地解释了随机奖励扰动何时及为何能产生良好的赌博机算法。我们提出了基于数据依赖的扰动机制（此为先前PHE类方法所不具备的），使EVILL在理论和实践上均能达到汤普森采样类参数扰动方法的性能。此外，我们展示了广义线性赌博机之外的一个实例：在该实例中PHE会导致估计不一致并产生线性遗憾，而EVILL仍能保持良好性能。与PHE相同，EVILL仅需数行代码即可实现。