The theory of learning in games has so far focused mainly on games with simultaneous moves. Recently, researchers in machine learning have started investigating learning dynamics in games involving hierarchical decision-making. We consider an $N$-player hierarchical game in which the $i$th player's objective comprises of an expectation-valued term, parametrized by rival decisions, and a hierarchical term. Such a framework allows for capturing a broad range of stochastic hierarchical optimization problems, Stackelberg equilibrium problems, and leader-follower games. We develop an iteratively regularized and smoothed variance-reduced modified extragradient framework for learning hierarchical equilibria in a stochastic setting. We equip our analysis with rate statements, complexity guarantees, and almost-sure convergence claims. We then extend these statements to settings where the lower-level problem is solved inexactly and provide the corresponding rate and complexity statements.

翻译：博弈学习理论迄今为止主要聚焦于同时行动的博弈。近期，机器学习研究者开始探究涉及层次化决策的博弈学习动态。我们考虑一个 $N$ 人层次博弈，其中第 $i$ 个玩家的目标包含一个以对手决策为参数的期望值项和一个层次项。这种框架能够刻画广泛的随机层次优化问题、Stackelberg均衡问题以及领导者-追随者博弈。我们发展了一种迭代正则化与平滑化的方差缩减修正外梯度框架，用于在随机环境中学习层次均衡。我们的分析配备了收敛速率陈述、复杂度保证以及几乎必然收敛结论。随后，我们将这些结论推广至下层问题非精确求解的场景，并给出相应的速率与复杂度分析。