Decision Trees are prominent prediction models for interpretable Machine Learning. They have been thoroughly researched, mostly in the batch setting with a fixed labelled dataset, leading to popular algorithms such as C4.5, ID3 and CART. Unfortunately, these methods are of heuristic nature, they rely on greedy splits offering no guarantees of global optimality and often leading to unnecessarily complex and hard-to-interpret Decision Trees. Recent breakthroughs addressed this suboptimality issue in the batch setting, but no such work has considered the online setting with data arriving in a stream. To this end, we devise a new Monte Carlo Tree Search algorithm, Thompson Sampling Decision Trees (TSDT), able to produce optimal Decision Trees in an online setting. We analyse our algorithm and prove its almost sure convergence to the optimal tree. Furthermore, we conduct extensive experiments to validate our findings empirically. The proposed TSDT outperforms existing algorithms on several benchmarks, all while presenting the practical advantage of being tailored to the online setting.

翻译：决策树是可解释机器学习中重要的预测模型。尽管这类模型已被广泛研究（主要集中于固定标注数据集的批量学习场景），并由此催生了C4.5、ID3和CART等经典算法，但这些方法本质上属于启发式算法，依赖贪婪分裂策略，既无法保证全局最优性，又常导致生成的决策树过于复杂且难以解释。尽管近期研究在批量学习场景下突破了这一次优性问题，但目前尚无工作关注数据流式到达的在线学习场景。为此，我们提出一种新型蒙特卡洛树搜索算法——汤普森采样决策树（TSDT），能够在在线学习场景中生成最优决策树。我们对该算法进行了理论分析，证明了其几乎必然收敛到最优树。此外，我们通过大量实验验证了理论发现。实验结果表明，所提出的TSDT在多个基准测试中均优于现有算法，同时具有专为在线场景设计的实用优势。