State of the art Symbolic Regression (SR) methods currently build specialized models, while the application of Large Language Models (LLMs) remains largely unexplored. In this work, we introduce the first comprehensive framework that utilizes LLMs for the task of SR. We propose In-Context Symbolic Regression (ICSR), an SR method which iteratively refines a functional form with an LLM and determines its coefficients with an external optimizer. ICSR leverages LLMs' strong mathematical prior both to propose an initial set of possible functions given the observations and to refine them based on their errors. Our findings reveal that LLMs are able to successfully find symbolic equations that fit the given data, matching or outperforming the overall performance of the best SR baselines on four popular benchmarks, while yielding simpler equations with better out of distribution generalization.

翻译：目前最先进的符号回归方法通常构建专用模型，而大语言模型在该领域的应用仍基本未被探索。本研究首次提出了利用大语言模型进行符号回归任务的完整框架。我们提出上下文符号回归方法，这是一种通过大语言模型迭代优化函数形式，并借助外部优化器确定其系数的符号回归技术。该方法充分利用大语言模型强大的数学先验知识：既可根据观测数据生成初始候选函数集，又能基于误差反馈进行函数优化。研究结果表明，大语言模型能够成功发现拟合给定数据的符号方程，在四个主流基准测试中达到或超越了最佳符号回归基线的整体性能，同时生成更简洁的方程并展现出更优异的分布外泛化能力。