Flexible Manufacturing Systems (FMS) are pivotal in optimizing production processes in today's rapidly evolving manufacturing landscape. This paper advances the traditional job shop scheduling problem by incorporating additional complexities through the simultaneous integration of automated guided vehicles (AGVs) and tool-sharing systems. We propose a novel approach that combines Colored-Timed Petri Nets (CTPNs) with actor-critic model-based reinforcement learning (MBRL), effectively addressing the multifaceted challenges associated with FMS. CTPNs provide a formal modeling structure and dynamic action masking, significantly reducing the action search space, while MBRL ensures adaptability to changing environments through the learned policy. Leveraging the advantages of MBRL, we incorporate a lookahead strategy for optimal positioning of AGVs, improving operational efficiency. Our approach was evaluated on small-sized public benchmarks and a newly developed large-scale benchmark inspired by the Taillard benchmark. The results show that our approach matches traditional methods on smaller instances and outperforms them on larger ones in terms of makespan while achieving a tenfold reduction in computation time. To ensure reproducibility, we propose a gym-compatible environment and an instance generator. Additionally, an ablation study evaluates the contribution of each framework component to its overall performance.

翻译：柔性制造系统（FMS）在当今快速发展的制造业环境中对优化生产过程至关重要。本文通过同时集成自动导引车（AGV）与刀具共享系统引入额外复杂性，从而推进了传统的作业车间调度问题。我们提出一种将着色时间Petri网（CTPN）与基于模型的演员-评论家强化学习（MBRL）相结合的新方法，有效应对FMS相关的多层面挑战。CTPN提供形式化建模结构与动态动作掩码机制，显著缩小动作搜索空间，而MBRL通过习得策略确保对动态环境的适应性。利用MBRL的优势，我们引入前瞻策略以实现AGV的最优定位，从而提升运行效率。我们在小型公共基准测试集及一个受Taillard基准启发新开发的大规模基准测试集上评估了所提方法。结果表明，在较小规模实例上我们的方法与传统方法性能相当，在较大规模实例上则在完工时间指标上优于传统方法，同时实现计算时间数量级的缩减。为确保可复现性，我们提出了兼容gym框架的环境与实例生成器。此外，消融实验评估了框架各组成部分对整体性能的贡献度。