In multi-agent reinforcement learning, decentralized execution is a common approach, yet it suffers from the redundant computation problem. This occurs when multiple agents redundantly perform the same or similar computation due to overlapping observations. To address this issue, this study introduces a novel method referred to as locally centralized team transformer (LCTT). LCTT establishes a locally centralized execution framework where selected agents serve as leaders, issuing instructions, while the rest agents, designated as workers, act as these instructions without activating their policy networks. For LCTT, we proposed the team-transformer (T-Trans) architecture that allows leaders to provide specific instructions to each worker, and the leadership shift mechanism that allows agents autonomously decide their roles as leaders or workers. Our experimental results demonstrate that the proposed method effectively reduces redundant computation, does not decrease reward levels, and leads to faster learning convergence.

翻译：在多智能体强化学习中，分散执行是一种常见方法，但存在冗余计算问题。当多个智能体因观测重叠而重复执行相同或相似计算时，就会产生该问题。为解决这一问题，本研究提出了一种名为局部中心化团队Transformer（LCTT）的新方法。LCTT建立了一个局部中心化执行框架：被选中的智能体作为领导者发布指令，其余被指定为工作者的智能体则直接执行这些指令，无需激活自身的策略网络。针对LCTT，我们提出了团队Transformer（T-Trans）架构，使领导者能够为每位工作者提供具体指令；同时设计了领导权转移机制，允许智能体自主决定其扮演领导或工作者的角色。实验结果表明，该方法能有效减少冗余计算，且不会降低奖励水平，同时促进学习收敛速度的提升。