We propose and analyze TRAiL (Tangential Randomization in Linear Bandits), a computationally efficient regret-optimal forced exploration algorithm for linear bandits on action sets that are sublevel sets of strongly convex functions. TRAiL estimates the governing parameter of the linear bandit problem through a standard regularized least squares and perturbs the reward-maximizing action corresponding to said point estimate along the tangent plane of the convex compact action set before projecting back to it. Exploiting concentration results for matrix martingales, we prove that TRAiL ensures a $\Omega(\sqrt{T})$ growth in the inference quality, measured via the minimum eigenvalue of the design (regressor) matrix with high-probability over a $T$-length period. We build on this result to obtain an $\mathcal{O}(\sqrt{T} \log(T))$ upper bound on cumulative regret with probability at least $ 1 - 1/T$ over $T$ periods, and compare TRAiL to other popular algorithms for linear bandits. Then, we characterize an $\Omega(\sqrt{T})$ minimax lower bound for any algorithm on the expected regret that covers a wide variety of action/parameter sets and noise processes. Our analysis not only expands the realm of lower-bounds in linear bandits significantly, but as a byproduct, yields a trade-off between regret and inference quality. Specifically, we prove that any algorithm with an $\mathcal{O}(T^\alpha)$ expected regret growth must have an $\Omega(T^{1-\alpha})$ asymptotic growth in expected inference quality. Our experiments on the $L^p$ unit ball as action sets reveal how this relation can be violated, but only in the short-run, before returning to respect the bound asymptotically. In effect, regret-minimizing algorithms must have just the right rate of inference -- too fast or too slow inference will incur sub-optimal regret growth.

翻译：我们提出并分析了TRAiL（线性赌博机中的切向随机化），这是一种计算高效的、遗憾最优的强制探索算法，适用于行动集为强凸函数次水平集的线性赌博机问题。TRAiL通过标准的正则化最小二乘法估计线性赌博机问题的控制参数，并将对应于该点估计的奖励最大化行动沿凸紧致行动集的切平面扰动，然后将其投影回行动集。利用矩阵鞅的集中性结果，我们证明TRAiL能够以高概率确保在长度为$T$的周期内，通过设计（回归）矩阵的最小特征值衡量的推断质量以$\Omega(\sqrt{T})$的速度增长。基于这一结果，我们得到了一个以至少$1 - 1/T$的概率在$T$个周期内累积遗憾的$\mathcal{O}(\sqrt{T} \log(T))$上界，并将TRAiL与其他流行的线性赌博机算法进行了比较。接着，我们刻画了对于任何算法在期望遗憾上的$\Omega(\sqrt{T})$极小极大下界，该下界涵盖了多种行动/参数集和噪声过程。我们的分析不仅显著扩展了线性赌博机中下界的范畴，而且作为一个副产品，揭示了遗憾与推断质量之间的权衡关系。具体而言，我们证明了任何具有$\mathcal{O}(T^\alpha)$期望遗憾增长率的算法，其期望推断质量的渐近增长率必须为$\Omega(T^{1-\alpha})$。我们在以$L^p$单位球作为行动集上的实验揭示了这一关系可能被违反，但仅限于短期运行，最终会渐近地遵循该界。实际上，遗憾最小化算法必须具有恰当的推断速率——推断过快或过慢都将导致次优的遗憾增长。