Symbolic regression (SR) aims to discover mathematical expressions from data, a task traditionally tackled using Genetic Programming (GP) through combinatorial search over symbolic structures. Latent Space Optimization (LSO) methods use neural encoders to map symbolic expressions into continuous spaces, transforming the combinatorial search into continuous optimization. SNIP (Meidani et al., 2024), a contrastive pre-training model inspired by CLIP, advances LSO by introducing a multi-modal approach: aligning symbolic and numeric encoders in a shared latent space to learn the phenotype-genotype mapping, enabling optimization in the numeric space to implicitly guide symbolic search. However, this relies on fine-grained cross-modal alignment, whereas literature on similar models like CLIP reveals that such an alignment is typically coarse-grained. In this paper, we investigate whether SNIP delivers on its promise of effective bi-modal optimization for SR. Our experiments show that: (1) cross-modal alignment does not improve during optimization, even as fitness increases, and (2) the alignment learned by SNIP is too coarse to efficiently conduct principled search in the symbolic space. These findings reveal that while multi-modal LSO holds significant potential for SR, effective alignment-guided optimization remains unrealized in practice, highlighting fine-grained alignment as a critical direction for future work.

翻译：符号回归旨在从数据中发现数学表达式，传统上通过遗传规划在符号结构上组合搜索来完成。潜在空间优化方法利用神经编码器将符号表达式映射到连续空间，将组合搜索转化为连续优化。SNIP（Meidani等人，2024）是一种受CLIP启发的对比预训练模型，通过引入多模态方法推进了LSO：在共享潜在空间中对齐符号编码器与数值编码器以学习表型-基因型映射，从而通过数值空间的优化隐式引导符号搜索。然而，这依赖于细粒度的跨模态对齐，而CLIP等类似模型的文献表明此类对齐通常是粗粒度的。本文探究SNIP是否真正实现了其声称的符号回归双模态优化。实验表明：（1）即使适应度提升，跨模态对齐在优化过程中并未改善；（2）SNIP学得的对齐过于粗糙，无法高效进行符号空间中的有原则搜索。这些发现揭示，尽管多模态LSO对符号回归具有显著潜力，但实际中有效的对齐引导优化尚未实现，由此凸显细粒度对齐是未来工作的关键方向。