We consider model selection for sequential decision making in stochastic environments with bandit feedback, where a meta-learner has at its disposal a pool of base learners, and decides on the fly which action to take based on the policies recommended by each base learner. Model selection is performed by regret balancing but, unlike the recent literature on this subject, we do not assume any prior knowledge about the base learners like candidate regret guarantees; instead, we uncover these quantities in a data-driven manner. The meta-learner is therefore able to leverage the realized regret incurred by each base learner for the learning environment at hand (as opposed to the expected regret), and single out the best such regret. We design two model selection algorithms operating with this more ambitious notion of regret and, besides proving model selection guarantees via regret balancing, we experimentally demonstrate the compelling practical benefits of dealing with actual regrets instead of candidate regret bounds.

翻译：我们考虑在具有赌博反馈的随机环境中进行序列决策的模型选择问题，其中元学习器拥有一组基础学习器，并根据每个基础学习器推荐的策略动态决定采取何种行动。模型选择通过遗憾平衡实现，但与近期关于该主题的文献不同，我们无需事先了解基础学习器的任何先验知识（如候选遗憾保证），而是以数据驱动的方式揭示这些量。因此，元学习器能够利用每个基础学习器在当前学习环境中实际产生的遗憾（而非期望遗憾），并甄别出其中最优的遗憾值。我们设计了两种基于这种更具挑战性的遗憾概念运行的模型选择算法，除了通过遗憾平衡证明模型选择保证外，我们通过实验证明了处理实际遗憾而非候选遗憾界所带来的显著实用优势。