We consider model selection for sequential decision making in stochastic environments with bandit feedback, where a meta-learner has at its disposal a pool of base learners, and decides on the fly which action to take based on the policies recommended by each base learner. Model selection is performed by regret balancing but, unlike the recent literature on this subject, we do not assume any prior knowledge about the base learners like candidate regret guarantees; instead, we uncover these quantities in a data-driven manner. The meta-learner is therefore able to leverage the realized regret incurred by each base learner for the learning environment at hand (as opposed to the expected regret), and single out the best such regret. We design two model selection algorithms operating with this more ambitious notion of regret and, besides proving model selection guarantees via regret balancing, we experimentally demonstrate the compelling practical benefits of dealing with actual regrets instead of candidate regret bounds.

翻译：我们考虑在随机环境中具有Bandit反馈的序列决策中的模型选择问题，其中元学习器拥有一个基础学习器池，并根据每个基础学习器推荐的策略实时决定采取何种行动。模型选择通过遗憾平衡方法实现，但与近期相关文献不同，我们无需假设任何关于基础学习器的先验知识（如候选遗憾保证），而是以数据驱动的方式揭示这些量。因此，元学习器能够利用每个基础学习器在当前学习环境中产生的实际遗憾（而非期望遗憾），并从中选出最佳遗憾值。我们设计了两种基于这一更具挑战性的遗憾概念的模型选择算法，除了通过遗憾平衡证明模型选择保证外，还通过实验证明了处理实际遗憾而非候选遗憾界所带来的显著实际优势。