We study finite-horizon two-player zero-sum differential games with one-sided payoff information ($G$), where the informed player (P1) knows the game payoff, while P2 only has a public belief over a finite set of possible payoffs. In this case, P1's Nash equilibrium (NE) behavioral strategy may control the release of the type information or even resort to manipulate P2's belief. Previous studies revealed an atomic structure of the NE of $G$ with general nonlinear dynamics and payoffs, leading to tractable NE approximation. Implementing such approximation schemes for real-time sub-game solving, however, has not been achieved, yet is desired for applications where sim-to-real gaps exist and robust control is required. This paper improves the computational efficiency of sub-game solving for P1 during $G$ with linear dynamics and quadratic losses. Specifically, we show that P1's NE computation can be formulated as a bi-level optimization problem where the outer level optimizes the "signaling" strategy, i.e., when and how to reveal information through control, and the inner level is a game-tree LQR that solves for the optimal closed-loop control. This bi-level problem is solved via an adjoint-enabled backpropagation scheme: A "backward" LQR pass is followed by a "forward" gradient descent pass for improving the signaling. We apply the proposed algorithm to approximate NEs for variants of a homing problem with a 8D state space, 2D action spaces, and a discrete time horizon of $K=10$. The algorithm achieves $\approx$10Hz sub-game solving, enabling robust game-theoretic planning under information asymmetry and random disturbances.
翻译:我们研究了具有单边收益信息($G$)的有限时域双人零和微分博弈,其中知情者(P1)知晓博弈收益,而P2仅拥有关于有限可能收益集合的公开信念。在此情形下,P1的纳什均衡(NE)行为策略可能控制类型信息的释放,甚至通过操纵P2的信念来获利。已有研究揭示了具有一般非线性动力学和收益结构的NE原子结构,由此衍生出可处理的NE近似方法。然而,实时子博弈求解的近似方案尚未实现,这在存在模拟-现实鸿沟且需要鲁棒控制的应用场景中尤为期待。本文针对线性动力学与二次损失函数情形下的子博弈求解计算效率进行了改进。具体而言,我们证明P1的NE计算可转化为双层优化问题:外层优化"信号传递"策略(即通过控制何时及如何揭示信息),内层为求解最优闭环控制的博弈树LQR。该双层问题通过伴随启用的反向传播方案求解:先执行"反向"LQR传递,再执行"正向"梯度下降传递以改进信号传递。我们将所提算法应用于具有8维状态空间、2维动作空间及离散时间步长$K=10$的归航问题变体NE近似,实现约10Hz的子博弈求解频率,从而能够在信息不对称和随机扰动条件下进行鲁棒的博弈论规划。