We consider the problem of learning a sparse rule model, a prediction model in the form of a sparse linear combination of rules, where a rule is an indicator function defined over a hyper-rectangle in the input space. Since the number of all possible such rules is extremely large, it has been computationally intractable to select the optimal set of active rules. In this paper, to solve this difficulty for learning the optimal sparse rule model, we propose Safe RuleFit (SRF). Our basic idea is to develop meta safe screening (mSS), which is a non-trivial extension of well-known safe screening (SS) techniques. While SS is used for screening out one feature, mSS can be used for screening out multiple features by exploiting the inclusion-relations of hyper-rectangles in the input space. SRF provides a general framework for fitting sparse rule models for regression and classification, and it can be extended to handle more general sparse regularizations such as group regularization. We demonstrate the advantages of SRF through intensive numerical experiments.

翻译：我们考虑学习稀疏规则模型的问题，这是一种以稀疏线性组合规则形式呈现的预测模型，其中规则是定义在输入空间超矩形上的指示函数。由于所有可能规则的数量极其庞大，选择最优的活跃规则集在计算上一直难以处理。本文为解决学习最优稀疏规则模型的这一难题，提出了安全规则拟合（SRF）。我们的核心思想是发展元安全筛选（mSS），这是对经典安全筛选（SS）技术的非平凡扩展。SS用于筛选单个特征，而mSS通过利用输入空间中超矩形的包含关系，能够同时筛选多个特征。SRF为回归和分类任务中的稀疏规则模型拟合提供了一个通用框架，并可扩展至处理更一般的稀疏正则化形式（如组正则化）。我们通过大量数值实验验证了SRF的优越性。