Many real-world systems can be described by mathematical models that are human-comprehensible, easy to analyze and help explain the system's behavior. Symbolic regression is a method that can automatically generate such models from data. Historically, symbolic regression has been predominantly realized by genetic programming, a method that evolves populations of candidate solutions that are subsequently modified by genetic operators crossover and mutation. However, this approach suffers from several deficiencies: it does not scale well with the number of variables and samples in the training data - models tend to grow in size and complexity without an adequate accuracy gain, and it is hard to fine-tune the model coefficients using just genetic operators. Recently, neural networks have been applied to learn the whole analytic model, i.e., its structure and the coefficients, using gradient-based optimization algorithms. This paper proposes a novel neural network-based symbolic regression method that constructs physically plausible models based on even very small training data sets and prior knowledge about the system. The method employs an adaptive weighting scheme to effectively deal with multiple loss function terms and an epoch-wise learning process to reduce the chance of getting stuck in poor local optima. Furthermore, we propose a parameter-free method for choosing the model with the best interpolation and extrapolation performance out of all the models generated throughout the whole learning process. We experimentally evaluate the approach on four test systems: the TurtleBot 2 mobile robot, the magnetic manipulation system, the equivalent resistance of two resistors in parallel, and the longitudinal force of the anti-lock braking system. The results clearly show the potential of the method to find parsimonious models that comply with the prior knowledge provided.

翻译：许多现实世界系统可由人类可理解、易于分析且有助于解释系统行为的数学模型描述。符号回归是一种能够从数据中自动生成此类模型的方法。历史上，符号回归主要通过遗传编程实现，该方法进化候选解种群，随后通过遗传算子（交叉和变异）对其进行修改。然而，这种方法存在若干缺陷：其可扩展性在训练数据的变量和样本数量增加时表现不佳——模型往往在规模和复杂度上增长，却无法获得足够的精度增益，且仅通过遗传算子难以对模型系数进行微调。近年来，神经网络已被应用于利用基于梯度的优化算法学习整个解析模型，即其结构和系数。本文提出一种基于神经网络的新型符号回归方法，该方法能够基于极小的训练数据集和系统的先验知识构建物理可信的模型。该方法采用自适应加权方案以有效处理多个损失函数项，并通过逐轮学习过程降低陷入局部劣质最优解的风险。此外，我们提出一种无参数方法，从整个学习过程中生成的所有模型中选择内插和外推性能最佳的模型。我们在四个测试系统上进行了实验评估：TurtleBot 2移动机器人、磁操控系统、两个并联电阻的等效电阻以及防抱死制动系统的纵向力。结果清晰表明了该方法在发现符合给定先验知识的简约模型方面的潜力。