N-agent ad hoc teamwork (NAHT) is a newly introduced challenge in multi-agent reinforcement learning, where controlled subteams of varying sizes must dynamically collaborate with varying numbers and types of unknown teammates without pre-coordination. The existing learning algorithm (POAM) considers only independent learning for its flexibility in dealing with a changing number of agents. However, independent learning fails to fully capture the inter-agent dynamics essential for effective collaboration. Based on our observation that transformers deal effectively with sequences with varying lengths and have been shown to be highly effective for a variety of machine learning problems, this work introduces a centralized, transformer-based method for N-agent ad hoc teamwork. Our proposed approach incorporates historical observations and actions of all controlled agents, enabling optimal responses to diverse and unseen teammates in partially observable environments. Empirical evaluation on a StarCraft II task demonstrates that MAT-NAHT outperforms POAM, achieving superior sample efficiency and generalization, without auxiliary agent-modeling objectives.

翻译：N智能体临时协作（NAHT）是多智能体强化学习领域新近提出的挑战，其要求规模可变的可控子团队在无预先协调的情况下，与数量及类型均未知的队友进行动态协作。现有学习算法（POAM）仅考虑独立学习以应对智能体数量变化带来的灵活性需求，但独立学习难以充分捕捉有效协作所必需的主体间动态关系。基于Transformer能有效处理变长序列且在各类机器学习问题上表现卓越的观察，本文提出一种基于Transformer的集中式N智能体临时协作方法。所提方法融合所有可控智能体的历史观测与动作序列，使其能在部分可观测环境中对多样且未知的队友产生最优响应。在《星际争霸II》任务上的实证评估表明，MAT-NAHT在无需辅助智能体建模目标的情况下，其样本效率与泛化能力均优于POAM算法。