In many classification models, data is discretized to better estimate its distribution. Existing discretization methods often target at maximizing the discriminant power of discretized data, while overlooking the fact that the primary target of data discretization in classification is to improve the generalization performance. As a result, the data tend to be over-split into many small bins since the data without discretization retain the maximal discriminant information. Thus, we propose a Max-Dependency-Min-Divergence (MDmD) criterion that maximizes both the discriminant information and generalization ability of the discretized data. More specifically, the Max-Dependency criterion maximizes the statistical dependency between the discretized data and the classification variable while the Min-Divergence criterion explicitly minimizes the JS-divergence between the training data and the validation data for a given discretization scheme. The proposed MDmD criterion is technically appealing, but it is difficult to reliably estimate the high-order joint distributions of attributes and the classification variable. We hence further propose a more practical solution, Max-Relevance-Min-Divergence (MRmD) discretization scheme, where each attribute is discretized separately, by simultaneously maximizing the discriminant information and the generalization ability of the discretized data. The proposed MRmD is compared with the state-of-the-art discretization algorithms under the naive Bayes classification framework on 45 machine-learning benchmark datasets. It significantly outperforms all the compared methods on most of the datasets.

翻译：在许多分类模型中，数据被离散化以更好地估计其分布。现有离散化方法通常以最大化离散化数据的判别能力为目标，却忽视了分类中数据离散化的主要目标是提升泛化性能。这导致数据容易被过度分割成许多小区间，因为未经离散化的数据保留了最大判别信息。因此，我们提出一种最大化判别信息与泛化能力的最大依赖-最小差异准则。具体而言，最大依赖准则最大化离散化数据与分类变量之间的统计依赖性，而最小差异准则则明确最小化给定离散化方案下训练数据与验证数据之间的JS散度。所提出的最大依赖-最小差异准则在理论上具有吸引力，但难以可靠估计属性与分类变量的高阶联合分布。为此，我们进一步提出一种更实用的方案——最大相关-最小差异离散化方法，该方法通过同时最大化离散化数据的判别信息与泛化能力，对每个属性进行独立离散化。我们在45个机器学习基准数据集上，将所提出的最大相关-最小差异方法与朴素贝叶斯分类框架下的最优离散化算法进行了比较，结果表明其在大多数数据集上显著优于所有对比方法。