Constraint Programming (CP) is a declarative programming paradigm that allows for modeling and solving combinatorial optimization problems, such as the Job-Shop Scheduling Problem (JSSP). While CP solvers manage to find optimal or near-optimal solutions for small instances, they do not scale well to large ones, i.e., they require long computation times or yield low-quality solutions. Therefore, real-world scheduling applications often resort to fast, handcrafted, priority-based dispatching heuristics to find a good initial solution and then refine it using optimization methods. This paper proposes a novel end-to-end approach to solving scheduling problems by means of CP and Reinforcement Learning (RL). In contrast to previous RL methods, tailored for a given problem by including procedural simulation algorithms, complex feature engineering, or handcrafted reward functions, our neural-network architecture and training algorithm merely require a generic CP encoding of some scheduling problem along with a set of small instances. Our approach leverages existing CP solvers to train an agent learning a Priority Dispatching Rule (PDR) that generalizes well to large instances, even from separate datasets. We evaluate our method on seven JSSP datasets from the literature, showing its ability to find higher-quality solutions for very large instances than obtained by static PDRs and by a CP solver within the same time limit.

翻译：约束编程（CP）是一种声明式编程范式，可用于对作业车间调度问题（JSSP）等组合优化问题进行建模与求解。虽然CP求解器能够为小规模问题实例找到最优解或接近最优的解，但它们在处理大规模问题时扩展性不佳，即需要较长的计算时间，或只能提供低质量的解。因此，实际调度应用常采用快速、手工设计的优先级调度启发式算法来获得良好的初始解，再通过优化方法进行改进。本文提出了一种基于CP与强化学习（RL）的端到端调度问题求解新方法。与以往需针对特定问题设计过程仿真算法、复杂特征工程或手工奖励函数的RL方法不同，我们的神经网络架构与训练算法仅需针对某个调度问题提供通用CP编码以及一组小规模实例。该方法利用现有CP求解器训练智能体学习优先级调度规则（PDR），该规则能很好地泛化到大规模实例，甚至包括来自不同数据集的情况。我们在文献中七个JSSP数据集上评估了该方法，结果表明与静态PDR及同等时间限制下的CP求解器相比，本方法能为超大规模实例找到更高质量的解决方案。