Transformer Semantic Genetic Programming (TSGP) is a semantic search approach that uses a pre-trained transformer model as a variation operator to generate offspring programs with high semantic similarity to a given parent. Unlike other semantic GP approaches that rely on fixed syntactic transformations, TSGP aims to learn diverse structural variations that lead to solutions with similar semantics. We find that a single transformer model trained on millions of programs is able to generalize across symbolic regression problems of varying dimension. Evaluated on 24 real-world and synthetic datasets, TSGP significantly outperforms standard GP, SLIM_GSGP, Deep Symbolic Regression, and Denoising Autoencoder GP, achieving an average rank of 1.58 across all benchmarks. Moreover, TSGP produces more compact solutions than SLIM_GSGP, despite its higher accuracy. In addition, the target semantic distance is able to effectively adjust the step size in the semantic space: small values enable consistent improvement in fitness but often lead to larger programs, while larger values promote faster convergence and compactness. Thus, the target semantic distance provides an effective mechanism for balancing exploration and exploitation.

翻译：Transformer语义遗传编程（TSGP）是一种语义搜索方法，利用预训练的Transformer模型作为变异算子，生成与给定父代具有高度语义相似性的子代程序。与其他依赖固定句法变换的语义遗传编程方法不同，TSGP旨在学习多样化的结构变异，从而获得语义相似的解。研究发现，单个在数百万程序上训练的Transformer模型能够泛化至不同维度的符号回归问题。在24个真实世界与合成数据集上的评估表明，TSGP显著优于标准遗传编程、SLIM_GSGP、深度符号回归及去噪自编码器遗传编程，在所有基准测试中平均排名第1.58。此外，尽管TSGP具有更高精度，其生成的解比SLIM_GSGP更紧凑。同时，目标语义距离能够有效调节语义空间中的步长：较小值可实现适应度的持续改进，但常导致程序规模增大；较大值则促进快速收敛与紧凑性。因此，目标语义距离为平衡探索与利用提供了有效机制。