Constraint programming is known for being an efficient approach for solving combinatorial problems. Important design choices in a solver are the branching heuristics, which are designed to lead the search to the best solutions in a minimum amount of time. However, developing these heuristics is a time-consuming process that requires problem-specific expertise. This observation has motivated many efforts to use machine learning to automatically learn efficient heuristics without expert intervention. To the best of our knowledge, it is still an open research question. Although several generic variable-selection heuristics are available in the literature, the options for a generic value-selection heuristic are more scarce. In this paper, we propose to tackle this issue by introducing a generic learning procedure that can be used to obtain a value-selection heuristic inside a constraint programming solver. This has been achieved thanks to the combination of a deep Q-learning algorithm, a tailored reward signal, and a heterogeneous graph neural network architecture. Experiments on graph coloring, maximum independent set, and maximum cut problems show that our framework is able to find better solutions close to optimality without requiring a large amounts of backtracks while being generic.

翻译：约束规划以高效解决组合问题而闻名。求解器中的关键设计之一是分支启发式算法，旨在以最短时间引导搜索找到最优解。然而，开发这些启发式算法需要耗费大量时间，且需具备特定领域专业知识。这一观察推动了众多利用机器学习自动学习高效启发式算法（无需人工干预）的研究尝试。据我们所知，这仍是一个开放性问题。尽管文献中已存在多种通用变量选择启发式算法，但通用值选择启发式算法的选项更为稀缺。本文通过引入一种通用学习程序来解决该问题，该程序可在约束规划求解器内部生成值选择启发式算法。这一成果得益于深度Q学习算法、定制化奖励信号以及异构图神经网络架构的结合。在图着色、最大独立集和最大割问题上的实验表明，我们的框架无需大量回溯即可在接近最优解的情况下找到更优解，同时保持其通用性。