Symbolic regression (SR) is the process of discovering hidden relationships from data with mathematical expressions, which is considered an effective way to reach interpretable machine learning (ML). Genetic programming (GP) has been the dominator in solving SR problems. However, as the scale of SR problems increases, GP often poorly demonstrates and cannot effectively address the real-world high-dimensional problems. This limitation is mainly caused by the stochastic evolutionary nature of traditional GP in constructing the trees. In this paper, we propose a differentiable approach named DGP to construct GP trees towards high-dimensional SR for the first time. Specifically, a new data structure called differentiable symbolic tree is proposed to relax the discrete structure to be continuous, thus a gradient-based optimizer can be presented for the efficient optimization. In addition, a sampling method is proposed to eliminate the discrepancy caused by the above relaxation for valid symbolic expressions. Furthermore, a diversification mechanism is introduced to promote the optimizer escaping from local optima for globally better solutions. With these designs, the proposed DGP method can efficiently search for the GP trees with higher performance, thus being capable of dealing with high-dimensional SR. To demonstrate the effectiveness of DGP, we conducted various experiments against the state of the arts based on both GP and deep neural networks. The experiment results reveal that DGP can outperform these chosen peer competitors on high-dimensional regression benchmarks with dimensions varying from tens to thousands. In addition, on the synthetic SR problems, the proposed DGP method can also achieve the best recovery rate even with different noisy levels. It is believed this work can facilitate SR being a powerful alternative to interpretable ML for a broader range of real-world problems.

翻译：符号回归（SR）是通过数学表达式从数据中发现隐藏关系的过程，被视为实现可解释机器学习（ML）的有效途径。遗传规划（GP）一直是解决SR问题的主导方法。然而，随着SR问题规模的增大，GP的表现往往不佳，无法有效应对现实世界中的高维问题。这一局限主要源于传统GP在构建树结构时采用的随机演化本质。本文首次提出了一种名为DGP的可微分方法，用于构建面向高维SR的GP树。具体而言，我们提出了一种称为可微分符号树的新数据结构，将离散结构松弛为连续形式，从而可采用基于梯度的优化器实现高效优化。此外，我们提出了一种采样方法，以消除上述松弛导致的合法符号表达式差异。同时，引入多样化机制促进优化器跳出局部最优，从而获得全局更优解。通过上述设计，所提出的DGP方法能够高效搜索性能更优的GP树，因而具备处理高维SR的能力。为验证DGP的有效性，我们基于GP和深度神经网络进行了多项与当前最优方法的对比实验。实验结果表明，在维度从数十到数千不等的高维回归基准测试中，DGP能够超越所选的对比方法。此外，在合成SR问题上，即便在不同噪声水平下，所提出的DGP方法也能获得最高的恢复率。我们相信，这项工作能够推动SR成为更广泛现实问题中可解释ML的有力替代方案。