We propose a new transformer model for the Traveling Salesman Problem (TSP) called CycleFormer. We identified distinctive characteristics that need to be considered when applying a conventional transformer model to TSP and aimed to fully incorporate these elements into the TSP-specific transformer. Unlike the token sets in typical language models, which are limited and static, the token (node) set in TSP is unlimited and dynamic. To exploit this fact to the fullest, we equated the encoder output with the decoder linear layer and directly connected the context vector of the encoder to the decoder encoding. Additionally, we added a positional encoding to the encoder tokens that reflects the two-dimensional nature of TSP, and devised a circular positional encoding for the decoder tokens that considers the cyclic properties of a tour. By incorporating these ideas, CycleFormer outperforms state-of-the-art (SOTA) transformer models for TSP from TSP-50 to TSP-500. Notably, on TSP-500, the optimality gap was reduced by approximately 2.8 times, from 3.09% to 1.10%, compared to the existing SOTA. The code will be made available at https://github.com/Giventicket/CycleFormer.

翻译：我们提出了一种用于旅行商问题（TSP）的新型Transformer模型，称为CycleFormer。我们识别了将传统Transformer模型应用于TSP时需要考虑的独特特性，并致力于将这些要素充分整合到TSP专用的Transformer中。与典型语言模型中有限且静态的标记集合不同，TSP中的标记（节点）集合是无限且动态的。为了充分利用这一事实，我们将编码器输出与解码器线性层等同起来，并将编码器的上下文向量直接连接到解码器编码。此外，我们在编码器标记中添加了反映TSP二维特性的位置编码，并为解码器标记设计了考虑路径循环特性的循环位置编码。通过整合这些思想，CycleFormer在TSP-50到TSP-500的规模上均优于最先进的（SOTA）TSP Transformer模型。值得注意的是，在TSP-500上，与现有SOTA模型相比，最优性间隙从3.09%降低至1.10%，减少了约2.8倍。代码将在https://github.com/Giventicket/CycleFormer 公开。