To address the needs of modeling uncertainty in sensitive machine learning applications, the setup of distributionally robust optimization (DRO) seeks good performance uniformly across a variety of tasks. The recent multi-distribution learning (MDL) framework tackles this objective in a dynamic interaction with the environment, where the learner has sampling access to each target distribution. Drawing inspiration from the field of pure-exploration multi-armed bandits, we provide distribution-dependent guarantees in the MDL regime, that scale with suboptimality gaps and result in superior dependence on the sample size when compared to the existing distribution-independent analyses. We investigate two non-adaptive strategies, uniform and non-uniform exploration, and present non-asymptotic regret bounds using novel tools from empirical process theory. Furthermore, we devise an adaptive optimistic algorithm, LCB-DR, that showcases enhanced dependence on the gaps, mirroring the contrast between uniform and optimistic allocation in the multi-armed bandit literature. We also conduct a small synthetic experiment illustrating the comparative strengths of each strategy.

翻译：为满足敏感机器学习应用中不确定性建模的需求，分布鲁棒优化（DRO）框架致力于在多种任务上实现一致的良好性能。近期提出的多分布学习（MDL）框架通过与环境进行动态交互来实现这一目标，其中学习者可以对每个目标分布进行采样。受纯探索多臂老虎机领域的启发，我们在MDL机制中提出了分布依赖的保证，其收敛速率随次优间隙而变化，与现有的分布无关分析相比，在样本量依赖性方面表现出更优越的特性。我们研究了两种非自适应策略——均匀探索与非均匀探索，并利用经验过程理论中的新工具给出了非渐近遗憾界。此外，我们设计了一种自适应乐观算法LCB-DR，该算法展现出对间隙更强的依赖性，这反映了多臂老虎机文献中均匀分配与乐观分配之间的对比关系。我们还通过一个小型合成实验说明了每种策略的比较优势。