Spatial information is essential in various fields. How to explicitly model according to the spatial location of agents is also very important for the multi-agent problem, especially when the number of agents is changing and the scale is enormous. Inspired by the point cloud task in computer vision, we propose a spatial information extraction structure for multi-agent reinforcement learning in this paper. Agents can effectively share the neighborhood and global information through a spatially encoder-decoder structure. Our method follows the centralized training with decentralized execution (CTDE) paradigm. In addition, our structure can be applied to various existing mainstream reinforcement learning algorithms with minor modifications and can deal with the problem with a variable number of agents. The experiments in several multi-agent scenarios show that the existing methods can get convincing results by adding our spatially explicit architecture.

翻译：空间信息在各领域中至关重要。如何根据智能体的空间位置进行显式建模，对于多智能体问题尤为关键，尤其是在智能体数量动态变化且规模庞大的场景中。受计算机视觉中点云任务的启发，本文提出了一种用于多智能体强化学习的空间信息提取结构。智能体通过空间编码器-解码器结构能够有效共享邻域及全局信息。该方法遵循集中式训练与分散式执行（CTDE）范式。此外，该结构仅需少量修改即可应用于现有主流强化学习算法，并能处理变数量智能体问题。在多个多智能体场景下的实验表明，通过引入本文提出的空间显式架构，现有方法能够取得具有说服力的结果。