Despite their remarkable effectiveness and broad application, the drivers of success underlying ensembles of trees are still not fully understood. In this paper, we highlight how interpreting tree ensembles as adaptive and self-regularizing smoothers can provide new intuition and deeper insight to this topic. We use this perspective to show that, when studied as smoothers, randomized tree ensembles not only make predictions that are quantifiably more smooth than the predictions of the individual trees they consist of, but also further regulate their smoothness at test-time based on the dissimilarity between testing and training inputs. First, we use this insight to revisit, refine and reconcile two recent explanations of forest success by providing a new way of quantifying the conjectured behaviors of tree ensembles objectively by measuring the effective degree of smoothing they imply. Then, we move beyond existing explanations for the mechanisms by which tree ensembles improve upon individual trees and challenge the popular wisdom that the superior performance of forests should be understood as a consequence of variance reduction alone. We argue that the current high-level dichotomy into bias- and variance-reduction prevalent in statistics is insufficient to understand tree ensembles -- because the prevailing definition of bias does not capture differences in the expressivity of the hypothesis classes formed by trees and forests. Instead, we show that forests can improve upon trees by three distinct mechanisms that are usually implicitly entangled. In particular, we demonstrate that the smoothing effect of ensembling can reduce variance in predictions due to noise in outcome generation, reduce variability in the quality of the learned function given fixed input data and reduce potential bias in learnable functions by enriching the available hypothesis space.

翻译：尽管随机森林在应用中表现出显著有效性与广泛适用性，但其成功的内在驱动机制仍未被完全理解。本文通过将树集成模型阐释为自适应且具备自正则化能力的平滑器，为这一课题提供新的理论直觉与深层洞察。基于该视角，我们证明：当以平滑器视角研究时，随机化树集成不仅能够产生比其组成单棵树更平滑的预测（该平滑性可被量化衡量），还能在测试阶段根据测试输入与训练输入的差异度动态调节自身平滑程度。首先，我们利用这一发现重新审视、完善并调和了近期关于森林成功的两种解释——通过客观测量树集成隐含的有效平滑程度，量化验证其理论推断的行为特征。继而，我们超越现有关于树集成如何改进单棵树的机制解释，挑战"森林优越性能应仅归因于方差缩减"的主流认知。我们认为，当前统计学中普遍采用的偏差-方差二元对立框架不足以理解树集成——因为偏差的现有定义无法捕捉树与森林形成的假设类在表达能力上的差异。为此，我们证明森林可通过三种通常隐含纠缠的独立机制改善单棵树性能：具体而言，集成带来的平滑效应能够：（1）因结果生成过程的噪声降低预测方差；（2）在固定输入数据下减少学习函数质量的波动性；（3）通过丰富可用假设空间降低可学习函数的潜在偏差。