A common method of generalizing binary to multi-class classification is the error correcting code (ECC). ECCs may be optimized in a number of ways, for instance by making them orthogonal. Here we test two types of orthogonal ECCs on seven different datasets using three types of binary classifier and compare them with three other multi-class methods: 1 vs. 1, one-versus-the-rest and random ECCs. The first type of orthogonal ECC, in which the codes contain no zeros, admits a fast and simple method of solving for the probabilities. Orthogonal ECCs are always more accurate than random ECCs as predicted by recent literature. Improvments in uncertainty coefficient (U.C.) range between 0.4--17.5% (0.004--0.139, absolute), while improvements in Brier score between 0.7--10.7%. Unfortunately, orthogonal ECCs are rarely more accurate than 1 vs. 1. Disparities are worst when the methods are paired with logistic regression, with orthogonal ECCs never beating 1 vs. 1. When the methods are paired with SVM, the losses are less significant, peaking at 1.5%, relative, 0.011 absolute in uncertainty coefficient and 6.5% in Brier scores. Orthogonal ECCs are always the fastest of the five multi-class methods when paired with linear classifiers. When paired with a piecewise linear classifier, whose classification speed does not depend on the number of training samples, classifications using orthogonal ECCs were always more accurate than the other methods and also faster than 1 vs. 1. Losses against 1 vs. 1 here were higher, peaking at 1.9% (0.017, absolute), in U.C. and 39% in Brier score. Gains in speed ranged between 1.1% and over 100%. Whether the speed increase is worth the penalty in accuracy will depend on the application.
翻译:将二分类推广到多分类的常用方法是纠错编码(ECC)。ECC可通过多种方式优化,例如使其具有正交性。本文使用三种二分类器,在七个不同数据集上测试了两类正交ECC,并与另外三种多类方法(一对一、一对多和随机ECC)进行了比较。第一类不含零元素的正交ECC采用了一种快速简便的概率求解方法。正交ECC的准确率始终高于随机ECC,这与近期文献预测一致。不确定性系数(U.C.)的提升幅度为0.4%至17.5%(绝对值为0.004至0.139),Brier分数的提升幅度为0.7%至10.7%。遗憾的是,正交ECC的准确率很少超过一对一方法。当与逻辑回归配合使用时,差异最为显著,正交ECC从未超越一对一方法。当与SVM配合使用时,准确性损失较小,不确定性系数相对损失最高为1.5%(绝对值为0.011),Brier分数损失最高为6.5%。与线性分类器配合使用时,正交ECC始终是五种多类方法中最快的。当与分段线性分类器配合时(其分类速度不依赖于训练样本数量),使用正交ECC的分类准确率始终高于其他方法,且速度也快于一对一方法。此时与一对一方法相比,准确率损失有所增大,U.C.最高损失为1.9%(绝对值为0.017),Brier分数最高损失为39%。速度提升幅度介于1.1%至100%以上。速度提升是否值得以准确率下降为代价,将取决于具体应用场景。