Algorithms for solving the linear classification problem have a long history, dating back at least to 1936 with linear discriminant analysis. For linearly separable data, many algorithms can obtain the exact solution to the corresponding 0-1 loss classification problem efficiently, but for data which is not linearly separable, it has been shown that this problem, in full generality, is NP-hard. Alternative approaches all involve approximations of some kind, such as the use of surrogates for the 0-1 loss (for example, the hinge or logistic loss), none of which can be guaranteed to solve the problem exactly. Finding an efficient, rigorously proven algorithm for obtaining an exact (i.e., globally optimal) solution to the 0-1 loss linear classification problem remains an open problem. By analyzing the combinatorial and incidence relations between hyperplanes and data points, we derive a rigorous construction algorithm, incremental cell enumeration (ICE), that can solve the 0-1 loss classification problem exactly in $O(N^{D+1})$. To the best of our knowledge, this is the first standalone algorithm-one that does not rely on general-purpose solvers-with rigorously proven guarantees for this problem. Moreover, we further generalize ICE to address the polynomial hypersurface classification problem in $O(N^{G+1})$ time, where $G$ is determined by both the data dimension and the polynomial hypersurface degree. The correctness of our algorithm is proved by the use of tools from the theory of hyperplane arrangements and oriented matroids. We demonstrate the effectiveness of our algorithm on real-world datasets, achieving optimal training accuracy for small-scale datasets and higher test accuracy on most datasets. Furthermore, our complexity analysis shows that the ICE algorithm offers superior computational efficiency compared with state-of-the-art branch-and-bound algorithm.

翻译：解决线性分类问题的算法历史悠久，至少可追溯至1936年的线性判别分析。对于线性可分数据，许多算法能高效地获得对应0-1损失分类问题的精确解；但对于非线性可分数据，该问题在完全一般性下已被证明是NP难的。现有替代方法均涉及某种近似，例如使用0-1损失的替代损失函数（如合页损失或逻辑损失），但均无法保证精确求解该问题。寻找一种高效且严格可证明的算法，以获得0-1损失线性分类问题的精确（即全局最优）解，仍然是一个悬而未决的难题。通过分析超平面与数据点之间的组合关联与关联关系，我们提出了一种严格构造算法——增量单元枚举（ICE），该算法能以$O(N^{D+1})$的时间复杂度精确求解0-1损失分类问题。据我们所知，这是首个针对该问题具有严格可证明保证的独立算法（即不依赖通用求解器）。此外，我们进一步将ICE推广至多项式超曲面分类问题，其时间复杂度为$O(N^{G+1})$，其中$G$由数据维度和多项式超曲面次数共同决定。我们运用超平面构型理论及定向拟阵理论中的工具证明了算法的正确性。通过在真实数据集上的实验，我们验证了算法的有效性：在小规模数据集上实现了最优训练精度，在多数数据集上获得了更高的测试精度。进一步的复杂度分析表明，与当前最先进的分支定界算法相比，ICE算法具有更优越的计算效率。