We present Neural Random Forest Imitation - a novel approach for transforming random forests into neural networks. Existing methods propose a direct mapping and produce very inefficient architectures. In this work, we introduce an imitation learning approach by generating training data from a random forest and learning a neural network that imitates its behavior. This implicit transformation creates very efficient neural networks that learn the decision boundaries of a random forest. The generated model is differentiable, can be used as a warm start for fine-tuning, and enables end-to-end optimization. Experiments on several real-world benchmark datasets demonstrate superior performance, especially when training with very few training examples. Compared to state-of-the-art methods, we significantly reduce the number of network parameters while achieving the same or even improved accuracy due to better generalization.

翻译：我们提出神经随机森林模仿——一种将随机森林转化为神经网络的新型方法。现有方法采用直接映射方式，导致生成的网络架构效率极低。本研究采用模仿学习方法，通过从随机森林生成训练数据，训练能够模仿其行为的神经网络。这种隐式转换能创建出学习随机森林决策边界的高效神经网络。生成模型具有可微性，可作为微调的热启动方案，并支持端到端优化。在多个真实世界基准数据集上的实验表明，该模型尤其在训练样本极少的情况下展现出卓越性能。与现有方法相比，我们在显著减少网络参数量的同时，凭借更优的泛化能力实现了同等甚至更高的准确率。