Decision trees are one of the most popular classifiers in the machine learning literature. While the most common decision tree learning algorithms treat data as a batch, numerous algorithms have been proposed to construct decision trees from a data stream. A standard training strategy involves augmenting the current tree by changing a leaf node into a split. Here we typically maintain counters in each leaf which allow us to determine the optimal split, and whether the split should be done. In this paper we focus on how to speed up the search for the optimal split when dealing with sparse binary features and a binary class. We focus on finding splits that have the approximately optimal information gain or Gini index. In both cases finding the optimal split can be done in $O(d)$ time, where $d$ is the number of features. We propose an algorithm that yields $(1 + α)$ approximation when using conditional entropy in amortized $O(α^{-1}(1 + m\log d) \log \log n)$ time, where $m$ is the number of 1s in a data point, and $n$ is the number of data points. Similarly, for Gini index, we achieve $(1 + α)$ approximation in amortized $O(α^{-1} + m \log d)$ time. Our approach is beneficial for sparse data where $m \ll d$. In our experiments we find almost-optimal splits efficiently, faster than the baseline, overperforming the theoretical approximation guarantees.

翻译：决策树是机器学习领域最流行的分类器之一。尽管最常见的决策树学习算法将数据视为批量处理，但已有大量算法被提出用于从数据流中构建决策树。一种标准的训练策略是通过将叶节点转换为分割来扩展当前树。通常，我们会在每个叶节点中维护计数器，以便确定最优分割以及是否应执行分割。本文重点关注在处理稀疏二元特征和二元类别时，如何加速寻找最优分割的过程。我们致力于寻找具有近似最优信息增益或基尼指数的分割。在这两种情况下，寻找最优分割的时间复杂度均为 $O(d)$，其中 $d$ 为特征数量。我们提出一种算法，在使用条件熵时，可在摊还 $O(α^{-1}(1 + m\log d) \log \log n)$ 时间内获得 $(1 + α)$ 近似解，其中 $m$ 为数据点中取值为1的特征数量，$n$ 为数据点数量。类似地，对于基尼指数，我们可在摊还 $O(α^{-1} + m \log d)$ 时间内实现 $(1 + α)$ 近似。我们的方法对 $m \ll d$ 的稀疏数据尤为有利。实验结果表明，我们的方法能高效地找到近似最优分割，其速度优于基线方法，且实际表现超越了理论近似保证。