Neural networks' expressiveness comes at the cost of complex, black-box models that often extrapolate poorly beyond the domain of the training dataset, conflicting with the goal of finding compact analytic expressions to describe scientific data. We introduce OccamNet, a neural network model that finds interpretable, compact, and sparse symbolic fits to data, \`a la Occam's razor. Our model defines a probability distribution over functions with efficient sampling and function evaluation. We train by sampling functions and biasing the probability mass toward better fitting solutions, backpropagating using cross-entropy matching in a reinforcement-learning loss. OccamNet can identify symbolic fits for a variety of problems, including analytic and non-analytic functions, implicit functions, and simple image classification, and can outperform state-of-the-art symbolic regression methods on real-world regression datasets. Our method requires a minimal memory footprint, fits complicated functions in minutes on a single CPU, and scales on a GPU.

翻译：神经网络的表达能力以牺牲复杂黑箱模型为代价，这类模型通常难以在训练数据集之外进行良好外推，这与寻找描述科学数据的紧凑解析表达式的目标相悖。我们提出OccamNet——一种能够对数据拟合可解释、紧凑且稀疏符号表达的神经网络模型，其思想源自奥卡姆剃刀原理。该模型通过高效采样和函数评估定义了函数的概率分布。训练过程中，我们通过采样函数并将概率质量偏向更优拟合解，利用强化学习损失中的交叉熵匹配进行反向传播。OccamNet可识别多种问题的符号拟合，包括解析与非解析函数、隐函数及简单图像分类，并在真实世界回归数据集上优于最先进的符号回归方法。该方法所需内存占用极小，可在单CPU上数分钟内完成复杂函数拟合，并支持GPU扩展。