We introduce IMP-MARL, an open-source suite of multi-agent reinforcement learning (MARL) environments for large-scale Infrastructure Management Planning (IMP), offering a platform for benchmarking the scalability of cooperative MARL methods in real-world engineering applications. In IMP, a multi-component engineering system is subject to a risk of failure due to its components' damage condition. Specifically, each agent plans inspections and repairs for a specific system component, aiming to minimise maintenance costs while cooperating to minimise system failure risk. With IMP-MARL, we release several environments including one related to offshore wind structural systems, in an effort to meet today's needs to improve management strategies to support sustainable and reliable energy systems. Supported by IMP practical engineering environments featuring up to 100 agents, we conduct a benchmark campaign, where the scalability and performance of state-of-the-art cooperative MARL methods are compared against expert-based heuristic policies. The results reveal that centralised training with decentralised execution methods scale better with the number of agents than fully centralised or decentralised RL approaches, while also outperforming expert-based heuristic policies in most IMP environments. Based on our findings, we additionally outline remaining cooperation and scalability challenges that future MARL methods should still address. Through IMP-MARL, we encourage the implementation of new environments and the further development of MARL methods.

翻译：我们推出IMP-MARL，这是一个面向大规模基础设施管理规划（IMP）的开源多智能体强化学习（MARL）环境套件，为合作式MARL方法在真实工程应用中的可扩展性评估提供基准测试平台。在IMP场景中，多组件工程系统因其组件的损伤状态面临失效风险。具体而言，每个智能体负责规划特定系统组件的检测与维修策略，在最小化维护成本的同时通过协作降低系统失效风险。通过IMP-MARL，我们发布了包含海上风电结构系统在内的多个环境，旨在满足当前对改进管理策略以支持可持续可靠能源系统的需求。基于支持多达100个智能体的IMP实际工程环境，我们开展基准测试，将当前最优合作式MARL方法的可扩展性与性能与基于专家经验的启发式策略进行对比。结果表明，相较于完全集中式或分散式强化学习方法，集中训练与分散执行的方法在智能体数量增加时展现出更优的可扩展性，且在大多数IMP环境中表现优于专家启发式策略。基于研究发现，我们进一步指出了未来MARL方法仍需解决的合作与可扩展性挑战。通过IMP-MARL，我们鼓励新环境的实现以及MARL方法的进一步发展。