We optimize finite horizon multi-agent reach-avoid Markov decision process (MDP) via \emph{local feedback policies}. The global feedback policy solution yields global optimality but its communication complexity, memory usage and computation complexity scale exponentially with the number of agents. We mitigate this exponential dependency by restricting the solution space to local feedback policies and show that local feedback policies are rank-one factorizations of global feedback policies, which provides a principled approach to reducing communication complexity and memory usage. Additionally, by demonstrating that multi-agent reach-avoid MDPs over local feedback policies has a potential game structure, we show that iterative best response is a tractable multi-agent learning scheme with guaranteed convergence to deterministic Nash equilibrium, and derive each agent's best response via multiplicative dynamic program (DP) over the joint state space. Numerical simulations across different MDPs and agent sets show that the peak memory usage and offline computation complexity are significantly reduced while the approximation error to the optimal global reach-avoid objective is maintained.

翻译：我们通过\textit{局部反馈策略}优化有限时域多智能体到达-规避马尔可夫决策过程（MDP）。全局反馈策略的解能实现全局最优性，但其通信复杂度、内存占用和计算复杂度随智能体数量呈指数增长。我们通过将解空间限制为局部反馈策略来缓解这种指数依赖，并证明局部反馈策略是全局反馈策略的秩一分解，这为降低通信复杂度和内存占用提供了有原则的方法。此外，通过证明基于局部反馈策略的多智能体到达-规避MDP具有势博弈结构，我们证明迭代最优响应是一种可处理的多智能体学习方案，且能保证收敛到确定性纳什均衡，并推导出每个智能体在联合状态空间上的乘法动态规划（DP）最优响应。不同MDP和智能体集合上的数值仿真表明，峰值内存占用和离线计算复杂度显著降低，同时保持了与全局最优到达-规避目标间的近似误差。