Algorithms for solving the linear classification problem have a long history, dating back at least to 1936 with linear discriminant analysis. For linearly separable data, many algorithms can obtain the exact solution to the corresponding 0-1 loss classification problem efficiently, but for data which is not linearly separable, it has been shown that this problem, in full generality, is NP-hard. Alternative approaches all involve approximations of some kind, including the use of surrogates for the 0-1 loss (for example, the hinge or logistic loss) or approximate combinatorial search, none of which can be guaranteed to solve the problem exactly. Finding efficient algorithms to obtain an exact i.e. globally optimal solution for the 0-1 loss linear classification problem with fixed dimension, remains an open problem. In research we report here, we detail the rigorous construction of a new algorithm, incremental cell enumeration (ICE), that can solve the 0-1 loss classification problem exactly in polynomial time. We prove correctness using concepts from the theory of hyperplane arrangements and oriented matroids. We demonstrate the effectiveness of this algorithm on synthetic and real-world datasets, showing optimal accuracy both in and out-of-sample, in practical computational time. We also empirically demonstrate how the use of approximate upper bound leads to polynomial time run-time improvements to the algorithm whilst retaining exactness. To our knowledge, this is the first, rigorously-proven polynomial time, practical algorithm for this long-standing problem.

翻译：解决线性分类问题的算法历史悠久，至少可追溯至1936年的线性判别分析。对于线性可分数据，许多算法能高效获得对应0-1损失分类问题的精确解，但针对线性不可分数据，该问题已被证明在完全一般化情况下是NP困难的。替代方法均涉及某种形式的近似，包括使用0-1损失的替代函数（例如合页损失或逻辑损失）或近似组合搜索，但这些方法均无法保证精确求解。在固定维度下寻找能精确（即全局最优）解决0-1损失线性分类问题的高效算法，至今仍是开放问题。本文所述研究中，我们详细构建了一种新算法——增量单元枚举法（ICE），该算法能在多项式时间内精确求解0-1损失分类问题。我们利用超平面排列与定向拟阵理论中的概念证明了其正确性。通过在合成数据集和真实世界数据集上的实验，我们验证了该算法在实用计算时间内训练样本内外的预测精度均达到最优，并进一步实证表明，近似上界的使用可在保持精确性的同时实现多项式时间运行效率的提升。据我们所知，这是首个针对这一长期未决问题、经严格证明的多项式时间实用算法。