Distributed decision-making in multi-agent systems presents difficult challenges for interactive behavior learning in both cooperative and competitive systems. To mitigate this complexity, MAIDRL presents a semi-centralized Dense Reinforcement Learning algorithm enhanced by agent influence maps (AIMs), for learning effective multi-agent control on StarCraft Multi-Agent Challenge (SMAC) scenarios. In this paper, we extend the DenseNet in MAIDRL and introduce semi-centralized Multi-Agent Dense-CNN Reinforcement Learning, MAIDCRL, by incorporating convolutional layers into the deep model architecture, and evaluate the performance on both homogeneous and heterogeneous scenarios. The results show that the CNN-enabled MAIDCRL significantly improved the learning performance and achieved a faster learning rate compared to the existing MAIDRL, especially on more complicated heterogeneous SMAC scenarios. We further investigate the stability and robustness of our model. The statistics reflect that our model not only achieves higher winning rate in all the given scenarios but also boosts the agent's learning process in fine-grained decision-making.

翻译：在多智能体系统中，分布式决策对合作与竞争系统中的交互行为学习提出了严峻挑战。为缓解这一复杂性，MAIDRL提出了一种半集中式稠密强化学习算法，通过智能体影响图（AIMs）增强，在星际争霸多智能体挑战（SMAC）场景中实现有效的多智能体控制。本文在MAIDRL的基础上扩展了DenseNet，引入卷积层至深度模型架构，提出半集中式多智能体稠密CNN强化学习（MAIDCRL），并评估其在同质与异质场景中的性能。结果表明，与现有MAIDRL相比，基于CNN的MAIDCRL显著提升了学习性能，实现了更快的收敛速度，尤其在更复杂的异质SMAC场景中表现突出。我们进一步探究了模型的稳定性与鲁棒性。统计数据显示，该模型在给定所有场景中不仅实现了更高的胜率，还加速了智能体在细粒度决策过程中的学习。