Transformer-based models show state-of-the-art performance even for large-scale Traveling Salesman Problems (TSPs). However, they are based on fully-connected attention models and suffer from large computational complexity and GPU memory usage. We propose a lightweight CNN-Transformer model based on a CNN embedding layer and partial self-attention. Our CNN-Transformer model is able to better learn spatial features from input data using a CNN embedding layer compared with the standard Transformer models. It also removes considerable redundancy in fully connected attention models using the proposed partial self-attention. Experiments show that the proposed model outperforms other state-of-the-art Transformer-based models in terms of TSP solution quality, GPU memory usage, and inference time. Our model consumes approximately 20% less GPU memory usage and has 45% faster inference time compared with other state-of-the-art Transformer-based models. Our code is publicly available at https://github.com/cm8908/CNN_Transformer3

翻译：基于Transformer的模型即使在大规模旅行商问题（TSPs）上也展现出最先进的性能。然而，这些模型依赖于全连接注意力机制，面临计算复杂度和GPU内存占用较高的问题。我们提出了一种基于CNN嵌入层和部分自注意力的轻量级CNN-Transformer模型。与标准Transformer模型相比，我们的CNN-Transformer模型能够利用CNN嵌入层更好地从输入数据中学习空间特征，并通过所提出的部分自注意力机制显著减少了全连接注意力模型中的冗余。实验表明，该模型在TSP求解质量、GPU内存占用和推理时间方面均优于其他基于Transformer的最先进模型。与其他基于Transformer的最先进模型相比，我们的模型GPU内存占用减少约20%，推理速度提升45%。我们的代码已在https://github.com/cm8908/CNN_Transformer3上公开。