In this study, we propose a novel approach for investigating optimization performance by flexible robot coordination in automated warehouses with multi-agent reinforcement learning (MARL)-based control. Automated systems using robots are expected to achieve efficient operations compared with manual systems in terms of overall optimization performance. However, the impact of overall optimization on performance remains unclear in most automated systems due to a lack of suitable control methods. Thus, we proposed a centralized training-and-decentralized execution MARL framework as a practical overall optimization control method. In the proposed framework, we also proposed a single shared critic, trained with global states and rewards, applicable to a case in which heterogeneous agents make decisions asynchronously. Our proposed MARL framework was applied to the task selection of material handling equipment through automated order picking simulation, and its performance was evaluated to determine how far overall optimization outperforms partial optimization by comparing it with other MARL frameworks and rule-based control methods.

翻译：在本研究中，我们提出了一种新颖方法，通过基于多智能体强化学习（MARL）控制的柔性机器人协调，探讨自动化仓库中的优化性能。采用机器人的自动化系统相较于人工系统，在总体优化性能方面有望实现更高效的操作。然而，由于缺乏合适的控制方法，大多数自动化系统中总体优化对性能的影响仍不明确。为此，我们提出了一种集中训练与分散执行的MARL框架，作为实用的总体优化控制方法。在该框架中，我们还提出了一个单一共享评论家（critic），该评论家基于全局状态和奖励进行训练，适用于异构智能体异步决策的场景。我们将所提出的MARL框架应用于通过自动化订单拣选模拟实现的物料搬运设备任务选择，并通过与其他MARL框架及基于规则的控制方法进行比较，评估其性能以确定总体优化在多大程度上优于局部优化。