Symbolic regression (SR) aims to discover mathematical expressions from data, a task traditionally tackled using Genetic Programming (GP) through combinatorial search over symbolic structures. Latent Space Optimization (LSO) methods use neural encoders to map symbolic expressions into continuous spaces, transforming the combinatorial search into continuous optimization. SNIP (Meidani et al., 2024), a contrastive pre-training model inspired by CLIP, advances LSO by introducing a multi-modal approach: aligning symbolic and numeric encoders in a shared latent space to learn the phenotype-genotype mapping, enabling optimization in the numeric space to implicitly guide symbolic search. However, this relies on fine-grained cross-modal alignment, whereas literature on similar models like CLIP reveals that such an alignment is typically coarse-grained. In this paper, we investigate whether SNIP delivers on its promise of effective bi-modal optimization for SR. Our experiments show that: (1) cross-modal alignment does not improve during optimization, even as fitness increases, and (2) the alignment learned by SNIP is too coarse to efficiently conduct principled search in the symbolic space. These findings reveal that while multi-modal LSO holds significant potential for SR, effective alignment-guided optimization remains unrealized in practice, highlighting fine-grained alignment as a critical direction for future work.

翻译：符号回归（SR）旨在从数据中发现数学表达式，传统上通过遗传规划（GP）对符号结构进行组合搜索来解决该任务。潜空间优化（LSO）方法利用神经网络编码器将符号表达式映射到连续空间，从而将组合搜索转化为连续优化。受CLIP启发的对比预训练模型SNIP（Meidani et al., 2024）通过引入多模态方法推进了LSO：在共享潜空间中对齐符号编码器与数值编码器，以学习表型-基因型映射，从而通过在数值空间中进行优化来间接引导符号搜索。然而，这依赖于细粒度的跨模态对齐，而关于CLIP等类似模型的文献表明，这种对齐通常是粗粒度的。本文探究SNIP是否兑现了其针对SR进行有效双模态优化的承诺。实验表明：（1）即使在适应度提高的过程中，跨模态对齐在优化期间并未改善；（2）SNIP学到的对齐过于粗糙，无法在符号空间高效执行有原则的搜索。这些发现揭示，尽管多模态LSO对SR具有显著潜力，但在实践中有效的对齐引导优化仍未实现，从而凸显细粒度对齐是未来研究的关键方向。