Learning to play zero-sum games is a fundamental problem in game theory and machine learning. While significant progress has been made in minimizing external regret in the self-play settings or with full-information feedback, real-world applications often force learners to play against unknown, arbitrary opponents and restrict learners to bandit feedback where only the payoff of the realized action is observable. In such challenging settings, it is well-known that $Ω(\sqrt{T})$ external regret is unavoidable (where T is the number of rounds). To overcome this barrier, we investigate adversarial learning in zero-sum games under bandit feedback, aiming to minimize the deficit against the maximin pure strategy -- a metric we term Pure-Strategy Maximin Regret. We analyze this problem under two bandit feedback models: uninformed (only the realized reward is revealed) and informed (both the reward and the opponent's action are revealed). For uninformed bandit learning of normal-form games, we show that the Tsallis-INF algorithm achieves $O(c \log T)$ instance-dependent regret with a game-dependent parameter $c$. Crucially, we prove an information-theoretic lower bound showing that the dependence on c is necessary. To overcome this hardness, we turn to the informed setting and introduce Maximin-UCB, which obtains another regret bound of the form $O(c' \log T)$ for a different game-dependent parameter $c'$ that could potentially be much smaller than $c$. Finally, we generalize both results to bilinear games over an arbitrary, large action set, proposing Tsallis-FTRL-SPM and Maximin-LinUCB for the uninformed and informed setting respectively and establishing similar game-dependent logarithmic regret bounds.

翻译：学习玩零和博弈是博弈论和机器学习中的一个基本问题。尽管在自对弈设置或全信息反馈下最小化外部遗憾方面已取得显著进展，但现实世界的应用往往迫使学习者与未知的任意对手博弈，并将学习者限制在强盗反馈下，即只能观察到已执行动作的收益。在这种具有挑战性的设置中，众所周知$Ω(\sqrt{T})$的外部遗憾是不可避免的（其中T为回合数）。为克服这一障碍，我们研究了强盗反馈下零和博弈中的对抗学习，旨在最小化相对于最大最小纯策略的差距——我们称之为纯策略最大最小遗憾。我们在两种强盗反馈模型下分析此问题：无信息型（仅揭示已实现奖励）和有信息型（同时揭示奖励和对手动作）。对于标准形式博弈的无信息强盗学习，我们证明Tsallis-INF算法实现了$O(c \log T)$的实例依赖遗憾，其中c为博弈依赖参数。关键的是，我们证明了一个信息论下界，表明对c的依赖是必要的。为克服这一困难，我们转向有信息设置，并引入Maximin-UCB算法，该算法获得了另一种形式为$O(c' \log T)$的遗憾界，其中博弈依赖参数$c'$可能远小于c。最后，我们将这两个结果推广到任意大动作集上的双线性博弈，分别针对无信息和有信息设置提出了Tsallis-FTRL-SPM和Maximin-LinUCB算法，并建立了类似的博弈依赖对数遗憾界。