Model selection in supervised learning provides costless guarantees as if the model that best balances bias and variance was known a priori. We study the feasibility of similar guarantees for cumulative regret minimization in the stochastic contextual bandit setting. Recent work [Marinov and Zimmert, 2021] identifies instances where no algorithm can guarantee costless regret bounds. Nevertheless, we identify benign conditions where costless model selection is feasible: gradually increasing class complexity, and diminishing marginal returns for best-in-class policy value with increasing class complexity. Our algorithm is based on a novel misspecification test, and our analysis demonstrates the benefits of using model selection for reward estimation. Unlike prior work on model selection in contextual bandits, our algorithm carefully adapts to the evolving bias-variance trade-off as more data is collected. In particular, our algorithm and analysis go beyond adapting to the complexity of the simplest realizable class and instead adapt to the complexity of the simplest class whose estimation variance dominates the bias. For short horizons, this provides improved regret guarantees that depend on the complexity of simpler classes.

翻译：监督学习中的模型选择提供了无需额外代价的保证，仿佛最佳平衡偏差与方差的模型是先验已知的。我们研究在随机上下文赌博机设定中，针对累积遗憾最小化实现类似保证的可行性。近期工作[Marinov and Zimmert, 2021]指出，在某些实例中，不存在能够提供无代价遗憾界的算法。尽管如此，我们识别出可实现无代价模型选择的良性条件：类别复杂性逐步增加，且随着类别复杂性提高，类别最优策略价值的边际回报递减。本文算法基于一种新颖的误设定检验，分析揭示了利用模型选择进行奖励估计的优势。与先前上下文赌博机中模型选择的研究不同，我们的算法会随数据积累而自适应地调整偏差-方差权衡。具体而言，本文算法与分析超越了对最简单可实现类别复杂性的适应性，转而自适应于估计方差主导偏差的最简单类别复杂性。在短时间范围内，这提供了依赖于更简单类别复杂性的改进遗憾保证。