Adapting to a priori unknown noise level is a very important but challenging problem in sequential decision-making as efficient exploration typically requires knowledge of the noise level, which is often loosely specified. We report significant progress in addressing this issue in linear bandits in two respects. First, we propose a novel confidence set that is `semi-adaptive' to the unknown sub-Gaussian parameter $\sigma_*^2$ in the sense that the (normalized) confidence width scales with $\sqrt{d\sigma_*^2 + \sigma_0^2}$ where $d$ is the dimension and $\sigma_0^2$ is the specified sub-Gaussian parameter (known) that can be much larger than $\sigma_*^2$. This is a significant improvement over $\sqrt{d\sigma_0^2}$ of the standard confidence set of Abbasi-Yadkori et al. (2011), especially when $d$ is large. We show that this leads to an improved regret bound in linear bandits. Second, for bounded rewards, we propose a novel variance-adaptive confidence set that has a much improved numerical performance upon prior art. We then apply this confidence set to develop, as we claim, the first practical variance-adaptive linear bandit algorithm via an optimistic approach, which is enabled by our novel regret analysis technique. Both of our confidence sets rely critically on `regret equality' from online learning. Our empirical evaluation in Bayesian optimization tasks shows that our algorithms demonstrate better or comparable performance compared to existing methods.

翻译：自适应于先验未知的噪声水平是序列决策中非常重要但具有挑战性的问题，因为高效探索通常需要噪声水平的知识，而这一参数往往难以精确指定。我们在线性赌博机中针对此问题取得了两方面的显著进展。首先，我们提出了一种新颖的置信集，该置信集对未知的次高斯参数$\sigma_*^2$具有“半自适应”性，即（归一化）置信宽度与$\sqrt{d\sigma_*^2 + \sigma_0^2}$成比例，其中$d$为维度，$\sigma_0^2$为指定的已知次高斯参数，其值可能远大于$\sigma_*^2$。相比于Abbasi-Yadkori等人（2011）标准置信集中的$\sqrt{d\sigma_0^2}$，这一改进在$d$较大时尤为显著。我们证明这在线性赌博机中可带来更优的遗憾界。其次，针对有界奖励，我们提出了一种新颖的方差自适应置信集，其数值性能较现有方法有大幅提升。随后，我们将此置信集应用于开发——据我们所称——首个基于乐观方法的实用方差自适应线性赌博机算法，这得益于我们创新的遗憾分析技术。我们的两种置信集均关键依赖于在线学习中的“遗憾等式”。在贝叶斯优化任务中的实证评估表明，我们的算法相较于现有方法展现出更优或相当的性能。