Optimal margin Distribution Machine (ODM) is a newly proposed statistical learning framework rooting in the novel margin theory, which demonstrates better generalization performance than the traditional large margin based counterparts. Nonetheless, it suffers from the ubiquitous scalability problem regarding both computation time and memory as other kernel methods. This paper proposes a scalable ODM, which can achieve nearly ten times speedup compared to the original ODM training method. For nonlinear kernels, we propose a novel distribution-aware partition method to make the local ODM trained on each partition be close and converge fast to the global one. When linear kernel is applied, we extend a communication efficient SVRG method to accelerate the training further. Extensive empirical studies validate that our proposed method is highly computational efficient and almost never worsen the generalization.

翻译：最优间隔分布机（ODM）是一种基于新型间隔理论提出的统计学习框架，其泛化性能优于传统基于大间隔的方法。然而，与其他核方法类似，ODM在计算时间和内存方面存在普遍的可扩展性问题。本文提出了一种可扩展的ODM，相较于原始ODM训练方法可实现近十倍的加速。针对非线性核，我们提出一种新颖的分布感知划分方法，使得在每个划分上训练的局部ODM能够接近全局ODM并快速收敛。当采用线性核时，我们扩展了一种通信高效的SVRG方法以进一步加速训练。大量实验结果表明，所提方法具有极高的计算效率，且几乎不会降低泛化性能。