Machine Learning-based heuristics have recently shown impressive performance in solving a variety of hard combinatorial optimization problems (COPs). However they generally rely on a separate neural model, specialized and trained for each single problem. Any variation of a problem requires adjustment of its model and re-training from scratch. In this paper, we propose GOAL (for Generalist combinatorial Optimization Agent Learning), a generalist model capable of efficiently solving multiple COPs and which can be fine-tuned to solve new COPs. GOAL consists of a single backbone plus light-weight problem-specific adapters, mostly for input and output processing. The backbone is based on a new form of mixed-attention blocks which allows to handle problems defined on graphs with arbitrary combinations of node, edge and instance-level features. Additionally, problems which involve heterogeneous nodes or edges, such as in multi-partite graphs, are handled through a novel multi-type transformer architecture, where the attention blocks are duplicated to attend only the relevant combination of types while relying on the same shared parameters. We train GOAL on a set of routing, scheduling and classic graph problems and show that it is only slightly inferior to the specialized baselines while being the first multi-task model that solves a variety of COPs. Finally, we showcase the strong transfer learning capacity of GOAL by fine-tuning or learning the adapters for new problems, with only few shots and little data.

翻译：基于机器学习的启发式方法在解决各类复杂组合优化问题方面近期展现出卓越性能。然而，这些方法通常依赖于为每个特定问题专门设计和训练的独立神经网络模型。问题的任何变动都需要调整对应模型并从头重新训练。本文提出GOAL（通用组合优化智能体学习器），这是一种能够高效求解多种组合优化问题的通用模型，并可通过微调适应新的组合优化问题。GOAL由单一主干网络和轻量级问题特定适配器构成，适配器主要用于输入输出处理。主干网络基于新型混合注意力模块，可处理定义在具有任意节点特征、边特征及实例级特征组合的图结构上的问题。此外，针对涉及异质节点或边的问题（如多部图），我们通过创新的多类型Transformer架构进行处理：该架构通过复制注意力模块使其仅关注相关类型组合，同时共享相同的参数。我们在路由规划、调度优化及经典图问题数据集上训练GOAL，结果表明其性能仅略低于专用基线模型，同时成为首个能求解多种组合优化问题的多任务模型。最后，我们通过少量样本和有限数据对新问题进行适配器微调或训练，展示了GOAL强大的迁移学习能力。