Bayesian Additive Regression Trees (BART) are a powerful ensemble learning technique for modeling nonlinear regression functions. Although initially BART was proposed for predicting only continuous and binary response variables, over the years multiple extensions have emerged that are suitable for estimating a wider class of response variables (e.g. categorical and count data) in a multitude of application areas. In this paper we describe a generalized framework for Bayesian trees and their additive ensembles where the response variable comes from an exponential family distribution and hence encompasses many prominent variants of BART. We derive sufficient conditions on the response distribution, under which the posterior concentrates at a minimax rate, up to a logarithmic factor. In this regard our results provide theoretical justification for the empirical success of BART and its variants. To support practitioners, we develop a Python package, also accessible in R via reticulate, that implements GBART for a range of exponential family response variables including Poisson, Inverse Gaussian, and Gamma distributions, alongside the standard continuous regression and binary classification settings. The package provides a user-friendly interface, enabling straightforward implementation of BART models across a broad class of response distributions.
翻译:贝叶斯加性回归树(BART)是一种用于建模非线性回归函数的强大集成学习技术。尽管最初BART仅针对连续型和二值响应变量进行预测,但随着时间推移,已涌现出多种适用于更广泛响应变量类型(如类别数据和计数数据)的扩展版本,并在众多应用领域得到推广。本文提出了一种针对贝叶斯树及其加性集成模型的广义框架,其中响应变量服从指数族分布,从而涵盖了BART的多个重要变体。我们推导了响应分布需满足的充分条件,使得后验分布能够以极小化最优收敛速率(至多相差对数因子)实现集中。就此而言,我们的研究结果为BART及其变体的实证成功提供了理论依据。为便于实践应用,我们开发了一个Python软件包(亦可通过reticulate工具在R环境中调用),该包实现了针对一系列指数族响应变量的广义BART(GBART),包括泊松分布、逆高斯分布和伽马分布,同时涵盖标准连续回归与二分类设定。该软件包提供用户友好型接口,使得BART模型能在广泛响应分布类别中便捷应用。