Despite the fast development of multi-agent systems (MAS) and multi-agent reinforcement learning (MARL) algorithms, there is a lack of unified evaluation platforms and commonly-acknowledged baseline implementation. Therefore, an urgent need is to develop an integrated library suite that delivers reliable MARL implementation and replicable evaluation in various benchmarks. To fill such a research gap, in this paper, we propose MARLlib, a comprehensive MARL algorithm library for solving multi-agent problems. With a novel design of agent-level distributed dataflow, MARLlib manages to unify tens of algorithms in a highly composable integration style. Moreover, MARLlib goes beyond current work by integrating diverse environment interfaces and providing flexible parameter sharing strategies; this allows for versatile solutions to cooperative, competitive, and mixed tasks with minimal code modifications for end users. Finally, MARLlib provides easy-to-use APIs and a fully decoupled configuration system to help end users manipulate the learning process. A plethora of experiments is conducted to substantiate the correctness of our implementation, based on which we further derive new insights into the relationship between the performance and the design of algorithmic components. With MARLlib, we expect researchers to be able to tackle broader real-world multi-agent problems with trustworthy solutions. Github: \url{https://github.com/Replicable-MARL/MARLlib

翻译：尽管多智能体系统（MAS）与多智能体强化学习（MARL）算法发展迅速，但目前仍缺乏统一的评估平台和公认的基准实现。因此，开发一个能够提供可靠MARL实现并在各类基准测试中进行可重复评估的集成库套件具有迫切需求。为填补这一研究空白，本文提出MARLlib——一个用于解决多智能体问题的综合性MARL算法库。通过创新的智能体级分布式数据流设计，MARLlib以高度可组合的集成方式成功统一了数十种算法。此外，MARLlib超越了现有研究工作，通过整合多样化的环境接口并提供灵活的参数共享策略，使得用户只需极少的代码修改即可为协作、竞争及混合任务提供通用解决方案。最后，MARLlib提供易用的API和完全解耦的配置系统，帮助用户操控学习过程。大量实验验证了实现的正确性，并基于此进一步推导出算法组件性能与设计之间关系的新见解。借助MARLlib，我们期望研究者能够以可信赖的方案应对更广泛的真实世界多智能体问题。Github：\url{https://github.com/Replicable-MARL/MARLlib}