An influential paper of Calvano et al. empirically demonstrated that Q-learning agents spontaneously collude when placed as sellers that compete on prices in a natural market model. More recent results of Fish et al. empirically demonstrated that similar collusion happens with commercial LLMs. We formally prove that such collusion can also happen with external-regret-minimizing agents. We identify a very general class of agents, which we term Domination-Avoiding agents, that provably do not collude in such markets. This class contains all Mean-Based agents and all internal-regret-minimizing agents, as well as others such as Multiplicative-Weight agents with variable learning rate and contextual variants thereof. More generally we show that, in any game, this class of agents is guaranteed to jointly learn to almost never play strategies that are eliminated by repeated elimination of purely dominated strategies.

翻译：Calvano等人一篇具有影响力的论文通过实验证明，在自然市场模型中作为卖方竞争定价时，Q学习主体会自发共谋。Fish等人更近期的实验结果证明，类似共谋也会出现在商业大语言模型中。我们正式证明，这类共谋同样可能发生在外部遗憾最小化主体中。我们识别了一类非常广泛的主体（称之为“避免支配主体”），此类主体可证明不会在此类市场中共谋。该类别包含所有基于均值主体和所有内部遗憾最小化主体，以及其他如变学习率乘法权重主体及其上下文变体。更一般地，我们证明在任何博弈中，此类主体保证能共同学会几乎从不采用通过重复剔除纯粹支配策略后被剔除的策略。