We study the problem of contextual combinatorial semi-bandits, where input contexts are mapped into subsets of size $m$ of a collection of $K$ possible actions. In each round, the learner observes the realized reward of the predicted actions. Motivated by prototypical applications of contextual bandits, we focus on the $s$-sparse regime where we assume that the sum of rewards is bounded by some value $s\ll K$. For example, in recommendation systems the number of products purchased by any customer is significantly smaller than the total number of available products. Our main result is for the $(ε,δ)$-PAC variant of the problem for which we design an algorithm that returns an $ε$-optimal policy with high probability using a sample complexity of $\tilde{O}((poly(K/m)+sm/ε^2) \log(|Π|/δ))$ where $Π$ is the underlying (finite) class and $s$ is the sparsity parameter. This bound improves upon known bounds for combinatorial semi-bandits whenever $s\ll K$, and in the regime where $s=O(1)$, the leading term is independent of $K$. Our algorithm is also computationally efficient given access to an ERM oracle for $Π$. Our framework generalizes the list multiclass classification problem with bandit feedback, which can be seen as a special case with binary reward vectors. In the special case of single-label classification corresponding to $s=m=1$, we prove an $O((K^7+1/ε^2)\log(|H|/δ))$ sample complexity bound, which improves upon recent results in this scenario. Additionally, we consider the regret minimization setting where data can be generated adversarially, and establish a regret bound of $\tilde O(|Π|+\sqrt{smT\log |Π|})$, extending the result of Erez et al. (2024) who consider the simpler single label classification setting.

翻译：我们研究上下文组合半赌博机问题，其中输入上下文被映射到包含$K$个可能动作的集合中大小为$m$的子集。在每一轮中，学习者观测到预测动作的已实现奖励。受上下文赌博机典型应用的启发，我们关注$s$-稀疏机制，即假设奖励总和受某个值$s\ll K$的约束。例如，在推荐系统中，任何客户购买的产品数量远小于可用产品总数。我们的主要结果针对该问题的$(ε,δ)$-PAC变体，为此我们设计了一种算法，能以高概率返回$ε$-最优策略，其样本复杂度为$\tilde{O}((poly(K/m)+sm/ε^2) \log(|Π|/δ))$，其中$Π$是底层（有限）类别，$s$是稀疏性参数。该界限在$s\ll K$时改进了组合半赌博机的已知界限，且在$s=O(1)$机制下，主导项与$K$无关。若可访问$Π$的ERM预言机，我们的算法也具有计算高效性。我们的框架推广了具有赌博机反馈的列表多类分类问题，该问题可视为二元奖励向量下的特例。在对应于$s=m=1$的单标签分类特例中，我们证明了$O((K^7+1/ε^2)\log(|H|/δ))$的样本复杂度界限，这改进了该场景下的近期结果。此外，我们考虑数据可对抗性生成的遗憾最小化设置，并建立了$\tilde O(|Π|+\sqrt{smT\log |Π|})$的遗憾界限，扩展了Erez等人（2024年）针对更简单的单标签分类设置的研究结果。