Under the uncoupled learning setup, the last-iterate convergence guarantee towards Nash equilibrium is shown to be impossible in many games. This work studies the last-iterate convergence guarantee in general games toward rationalizability, a key solution concept in epistemic game theory that relaxes the stringent belief assumptions in both Nash and correlated equilibrium. This learning task naturally generalizes best arm identification problems, due to the intrinsic connections between rationalizable action profiles and the elimination of iteratively dominated actions. Despite a seemingly simple task, our first main result is a surprisingly negative one; that is, a large and natural class of no regret algorithms, including the entire family of Dual Averaging algorithms, provably take exponentially many rounds to reach rationalizability. Moreover, algorithms with the stronger no swap regret also suffer similar exponential inefficiency. To overcome these barriers, we develop a new algorithm that adjusts Exp3 with Diminishing Historical rewards (termed Exp3-DH); Exp3-DH gradually forgets history at carefully tailored rates. We prove that when all agents run Exp3-DH (a.k.a., self-play in multi-agent learning), all iteratively dominated actions can be eliminated within polynomially many rounds. Our experimental results further demonstrate the efficiency of Exp3-DH, and that state-of-the-art bandit algorithms, even those developed specifically for learning in games, fail to reach rationalizability efficiently.

翻译：在解耦学习框架下，诸多博弈中最后一轮迭代收敛至纳什均衡的保证已被证明不可能实现。本文研究一般博弈中最后一轮迭代收敛至可理性化的保证——这是认知博弈论中一个关键解概念，它放松了纳什均衡和相关均衡中对信念的严格假设。由于可理性化行动剖面与迭代剔除占优行动之间存在内在联系，该学习任务自然推广了最优臂识别问题。尽管任务看似简单，我们的首个主要结果却出人意料地消极：一类广泛且自然的无遗憾算法（包括整个对偶平均算法家族）被证明需要指数级轮次才能达到可理性化。此外，具有更强无交换遗憾的算法同样面临指数级效率瓶颈。为克服这些障碍，我们开发了一种新算法——基于衰减历史奖励的Exp3改进版（称为Exp3-DH）；Exp3-DH以精心设计的速率逐步遗忘历史信息。我们证明，当所有智能体均运行Exp3-DH（即多智能体学习中的自博弈）时，所有迭代占优行动可在多项式轮次内被剔除。实验结果进一步表明Exp3-DH的高效性，而现有最先进的赌博机算法（即便专为博弈学习设计）也无法高效达成可理性化。