The theory of learning in games has so far focused mainly on games with simultaneous moves. Recently, researchers in machine learning have started investigating learning dynamics in games involving hierarchical decision-making. We consider an $N$-player hierarchical game in which the $i$th player's objective comprises of an expectation-valued term, parametrized by rival decisions, and a hierarchical term. Such a framework allows for capturing a broad range of stochastic hierarchical optimization problems, Stackelberg equilibrium problems, and leader-follower games. We develop an iteratively regularized and smoothed variance-reduced modified extragradient framework for learning hierarchical equilibria in a stochastic setting. We equip our analysis with rate statements, complexity guarantees, and almost-sure convergence claims. We then extend these statements to settings where the lower-level problem is solved inexactly and provide the corresponding rate and complexity statements.

翻译：博弈学习理论迄今为止主要关注同时移动博弈。近期，机器学习研究者开始探索涉及分层决策的博弈学习动态。我们考虑一个$N$玩家分层博弈，其中第$i$个玩家的目标函数由期望值项（以对手决策为参数）和分层项组成。该框架能够涵盖广泛的随机分层优化问题、Stackelberg均衡问题及领导者-追随者博弈。我们提出了一种迭代正则化和平滑的方差缩减修正外梯度框架，用于在随机环境下学习分层均衡。我们为分析配备了速率陈述、复杂度保证和几乎必然收敛断言。随后我们将这些结论扩展至下层问题非精确求解的场景，并给出相应的速率与复杂度陈述。