In this paper, we examine the convergence landscape of multi-agent learning under uncertainty. Specifically, we analyze two stochastic models of regularized learning in continuous games -- one in continuous and one in discrete time with the aim of characterizing the long-run behavior of the induced sequence of play. In stark contrast to deterministic, full-information models of learning (or models with a vanishing learning rate), we show that the resulting dynamics do not converge in general. In lieu of this, we ask instead which actions are played more often in the long run, and by how much. We show that, in strongly monotone games, the dynamics of regularized learning may wander away from equilibrium infinitely often, but they always return to its vicinity in finite time (which we estimate), and their long-run distribution is sharply concentrated around a neighborhood thereof. We quantify the degree of this concentration, and we show that these favorable properties may all break down if the underlying game is not strongly monotone -- underscoring in this way the limits of regularized learning in the presence of persistent randomness and uncertainty.

翻译：本文研究了不确定性条件下多智能体学习的收敛性特征。具体而言，我们分析了连续博弈中两种正则化学习的随机模型——一种基于连续时间，另一种基于离散时间，旨在刻画博弈序列的长期行为特征。与确定性、完全信息的学习模型（或学习率衰减的模型）形成鲜明对比的是，我们证明所生成的动态过程通常不具备收敛性。鉴于此，我们转而探究哪些行动在长期运行中被更频繁地采用，及其频率差异。研究表明，在强单调博弈中，正则化学习的动态过程可能无限次偏离均衡点，但总能在有限时间内（我们估算了该时间）返回其邻域，且其长期分布会高度集中于该邻域范围内。我们量化了这种集中程度，并证明若底层博弈不具备强单调性，这些优良特性可能全部失效——这揭示了在持续随机性与不确定性条件下正则化学习的局限性。