Recently, Deep reinforcement learning (DRL) models have shown promising results in solving routing problems. However, most DRL solvers are commonly proposed to solve node routing problems, such as the Traveling Salesman Problem (TSP). Meanwhile, there has been limited research on applying neural methods to arc routing problems, such as the Chinese Postman Problem (CPP), since they often feature irregular and complex solution spaces compared to TSP. To fill these gaps, this paper proposes a novel DRL framework to address the CPP with load-dependent costs (CPP-LC) (Corberan et al., 2018), which is a complex arc routing problem with load constraints. The novelty of our method is two-fold. First, we formulate the CPP-LC as a Markov Decision Process (MDP) sequential model. Subsequently, we introduce an autoregressive model based on DRL, namely Arc-DRL, consisting of an encoder and decoder to address the CPP-LC challenge effectively. Such a framework allows the DRL model to work efficiently and scalably to arc routing problems. Furthermore, we propose a new bio-inspired meta-heuristic solution based on Evolutionary Algorithm (EA) for CPP-LC. Extensive experiments show that Arc-DRL outperforms existing meta-heuristic methods such as Iterative Local Search (ILS) and Variable Neighborhood Search (VNS) proposed by (Corberan et al., 2018) on large benchmark datasets for CPP-LC regarding both solution quality and running time; while the EA gives the best solution quality with much more running time. We release our C++ implementations for metaheuristics such as EA, ILS and VNS along with the code for data generation and our generated data at https://github.com/HySonLab/Chinese_Postman_Problem

翻译：近年来，深度强化学习模型在求解路径规划问题方面展现出了良好的效果。然而，大多数深度强化学习求解器通常用于求解节点路径规划问题，如旅行商问题。同时，由于弧路径规划问题（如中国邮路问题）相较于旅行商问题通常具有不规则且复杂的解空间，目前将神经网络方法应用于此类问题的研究较为有限。为填补这一空白，本文提出了一种新颖的深度强化学习框架，用于求解带负载依赖成本的中国邮路问题，这是带负载约束的复杂弧路径规划问题。我们方法的新颖性体现在两个方面：首先，我们将该问题建模为马尔可夫决策过程的序列模型；其次，我们引入了一种基于深度强化学习的自回归模型（命名为Arc-DRL），该模型由编码器和解码器组成，能够有效应对该问题的挑战。这一框架使得深度强化学习模型能够高效且可扩展地处理弧路径规划问题。此外，我们还提出了一种基于进化算法的生物启发式元启发式求解方法。大量实验表明，在大型基准数据集上，Arc-DRL在解质量和运行时间方面均优于现有元启发式方法（如迭代局部搜索和变邻域搜索）；而进化算法虽能获得最佳解质量，但所需运行时间显著增加。我们将进化算法、迭代局部搜索和变邻域搜索等元启发式算法的C++实现代码、数据生成代码以及生成的数据集发布在https://github.com/HySonLab/Chinese_Postman_Problem。