We consider model selection for sequential decision making in stochastic environments with bandit feedback, where a meta-learner has at its disposal a pool of base learners, and decides on the fly which action to take based on the policies recommended by each base learner. Model selection is performed by regret balancing but, unlike the recent literature on this subject, we do not assume any prior knowledge about the base learners like candidate regret guarantees; instead, we uncover these quantities in a data-driven manner. The meta-learner is therefore able to leverage the realized regret incurred by each base learner for the learning environment at hand (as opposed to the expected regret), and single out the best such regret. We design two model selection algorithms operating with this more ambitious notion of regret and, besides proving model selection guarantees via regret balancing, we experimentally demonstrate the compelling practical benefits of dealing with actual regrets instead of candidate regret bounds.

翻译：我们考虑随机环境下带有Bandit反馈的序贯决策中的模型选择问题。在该问题中，元学习器拥有一个基础学习器池，并基于每个基础学习器推荐策略实时决定采取何种行动。模型选择通过遗憾平衡实现，但与近期相关文献不同，我们并不假设对基础学习器具有先验知识（如候选遗憾保证），而是以数据驱动的方式揭示这些量值。因此，元学习器能够利用每个基础学习器在当前学习环境中实际产生的遗憾（而非期望遗憾），并甄选出最优遗憾值。我们设计了两种基于这种更宏大的遗憾概念运作的模型选择算法，不仅通过遗憾平衡证明了模型选择保证，更通过实验展示了处理实际遗憾而非候选遗憾界所带来的显著实践优势。