Bayesian Decision Trees (DTs) are generally considered a more advanced and accurate model than a regular Decision Tree (DT) because they can handle complex and uncertain data. Existing work on Bayesian DTs uses Markov Chain Monte Carlo (MCMC) with an accept-reject mechanism and sample using naive proposals to proceed to the next iteration, which can be slow because of the burn-in time needed. We can reduce the burn-in period by proposing a more sophisticated way of sampling or by designing a different numerical Bayesian approach. In this paper, we propose a replacement of the MCMC with an inherently parallel algorithm, the Sequential Monte Carlo (SMC), and a more effective sampling strategy inspired by the Evolutionary Algorithms (EA). Experiments show that SMC combined with the EA can produce more accurate results compared to MCMC in 100 times fewer iterations.
翻译:贝叶斯决策树通常被认为是比常规决策树更先进且更精确的模型,因为它能够处理复杂和不确定的数据。现有的贝叶斯决策树研究采用马尔可夫链蒙特卡洛方法,并借助接受-拒绝机制,通过简单提议来采样以进入下一次迭代,但由于所需的燃烧期时间,这一过程可能较为缓慢。我们可以通过提出更复杂的采样方式或设计不同的数值贝叶斯方法来缩短燃烧期。本文提出用天生并行的算法——序贯蒙特卡洛方法替代马尔可夫链蒙特卡洛方法,并结合受进化算法启发的更高效采样策略。实验表明,与马尔可夫链蒙特卡洛方法相比,序贯蒙特卡洛方法结合进化算法仅需百分之一的迭代次数即可产生更精确的结果。