Symbolic regression (SR) aims to discover mathematical expressions from data, a task traditionally tackled using Genetic Programming (GP) through combinatorial search over symbolic structures. Latent Space Optimization (LSO) methods use neural encoders to map symbolic expressions into continuous spaces, transforming the combinatorial search into continuous optimization. SNIP (Meidani et al., 2024), a contrastive pre-training model inspired by CLIP, advances LSO by introducing a multi-modal approach: aligning symbolic and numeric encoders in a shared latent space to learn the phenotype-genotype mapping, enabling optimization in the numeric space to implicitly guide symbolic search. However, this relies on fine-grained cross-modal alignment, whereas literature on similar models like CLIP reveals that such an alignment is typically coarse-grained. In this paper, we investigate whether SNIP delivers on its promise of effective bi-modal optimization for SR. Our experiments show that: (1) cross-modal alignment does not improve during optimization, even as fitness increases, and (2) the alignment learned by SNIP is too coarse to efficiently conduct principled search in the symbolic space. These findings reveal that while multi-modal LSO holds significant potential for SR, effective alignment-guided optimization remains unrealized in practice, highlighting fine-grained alignment as a critical direction for future work.

翻译：符号回归旨在从数据中发现数学表达式，传统上通过遗传编程对符号结构进行组合搜索来完成。潜空间优化方法利用神经编码器将符号表达式映射到连续空间，从而将组合搜索转化为连续优化。受CLIP启发的对比预训练模型SNIP通过引入多模态方法推进了潜空间优化：在共享潜空间中对齐符号编码器和数值编码器，以学习表型-基因型映射，从而在数值空间中进行优化，隐式引导符号搜索。然而，这依赖于细粒度的跨模态对齐，而关于CLIP等类似模型的文献表明，这种对齐通常是粗粒度的。本文研究了SNIP是否真正实现了符号回归中双模态优化的预期效果。实验结果表明：（1）交叉模态对齐在优化过程中并未改善，即便适应度提高；（2）SNIP学习到的对齐过于粗糙，无法有效在符号空间中进行原则性搜索。这些发现表明，尽管多模态潜空间优化在符号回归领域具有巨大潜力，但有效对齐引导的优化在实际中仍未实现，突出了细粒度对齐作为未来工作的关键方向。