Combinatorial optimization problems require an exhaustive search to find the optimal solution. A convenient approach to solving combinatorial optimization tasks in the form of Mixed Integer Linear Programs is Branch-and-Bound. Branch-and-Bound solver splits a task into two parts dividing the domain of an integer variable, then it solves them recursively, producing a tree of nested sub-tasks. The efficiency of the solver depends on the branchning heuristic used to select a variable for splitting. In the present work, we propose a reinforcement learning method that can efficiently learn the branching heuristic. We view the variable selection task as a tree Markov Decision Process, prove that the Bellman operator adapted for the tree Markov Decision Process is contracting in mean, and propose a modified learning objective for the reinforcement learning agent. Our agent requires less training data and produces smaller trees compared to previous reinforcement learning methods.

翻译：组合优化问题需要穷举搜索以找到最优解。处理混合整数线性规划形式下组合优化任务的一种便捷方法是分支定界法。分支定界求解器将任务拆分为两部分，划分整数变量的定义域，然后递归求解它们，生成嵌套子任务树。求解器的效率取决于用于选择分裂变量的分支启发式方法。在本文中，我们提出一种能够高效学习分支启发式方法的强化学习方法。我们将变量选择任务视为树形马尔可夫决策过程，证明了针对树形马尔可夫决策过程改进的贝尔曼算子在均值意义上具有收缩性，并为强化学习智能体提出了一种修改后的学习目标。与以往的强化学习方法相比，我们的智能体所需训练数据更少，生成的树规模也更小。