We introduce IMP-MARL, an open-source suite of multi-agent reinforcement learning (MARL) environments for large-scale Infrastructure Management Planning (IMP), offering a platform for benchmarking the scalability of cooperative MARL methods in real-world engineering applications. In IMP, a multi-component engineering system is subject to a risk of failure due to its components' damage condition. Specifically, each agent plans inspections and repairs for a specific system component, aiming to minimise maintenance costs while cooperating to minimise system failure risk. With IMP-MARL, we release several environments including one related to offshore wind structural systems, in an effort to meet today's needs to improve management strategies to support sustainable and reliable energy systems. Supported by IMP practical engineering environments featuring up to 100 agents, we conduct a benchmark campaign, where the scalability and performance of state-of-the-art cooperative MARL methods are compared against expert-based heuristic policies. The results reveal that centralised training with decentralised execution methods scale better with the number of agents than fully centralised or decentralised RL approaches, while also outperforming expert-based heuristic policies in most IMP environments. Based on our findings, we additionally outline remaining cooperation and scalability challenges that future MARL methods should still address. Through IMP-MARL, we encourage the implementation of new environments and the further development of MARL methods.

翻译：本文介绍了IMP-MARL，这是一个面向大规模基础设施管理规划（IMP）的开源多智能体强化学习（MARL）环境套件，为评估合作型MARL方法在实际工程应用中的可扩展性提供了基准平台。在IMP场景中，多组件工程系统因各构件损伤状态而面临失效风险。具体而言，每个智能体负责为其管理组件制定检测与维修计划，目标是在最小化维护成本的同时通过协同合作降低系统失效风险。为满足当前对可持续可靠能源系统管理策略优化的需求，IMP-MARL发布了多个环境，其中包括一个与海上风电结构系统相关的环境。基于支持多达100个智能体的IMP实际工程环境，我们开展了基准测试，将当前最先进合作型MARL方法的可扩展性与性能表现与基于专家知识的启发式策略进行对比。结果表明，采用集中训练分散执行模式的方法在智能体数量增加时具有优于完全集中或分散式强化学习方法的可扩展性，且在绝大多数IMP环境中的表现也优于基于专家知识的启发式策略。基于研究结论，我们进一步指出了未来MARL方法仍需解决的合作与可扩展性挑战。通过IMP-MARL，我们鼓励新环境的开发与MARL方法的持续创新。