Optimal margin Distribution Machine (ODM) is a newly proposed statistical learning framework rooting in the novel margin theory, which demonstrates better generalization performance than the traditional large margin based counterparts. Nonetheless, it suffers from the ubiquitous scalability problem regarding both computation time and memory as other kernel methods. This paper proposes a scalable ODM, which can achieve nearly ten times speedup compared to the original ODM training method. For nonlinear kernels, we propose a novel distribution-aware partition method to make the local ODM trained on each partition be close and converge fast to the global one. When linear kernel is applied, we extend a communication efficient SVRG method to accelerate the training further. Extensive empirical studies validate that our proposed method is highly computational efficient and almost never worsen the generalization.

翻译：最优边际分布机（ODM）是一种基于新边际理论提出的统计学习框架，与传统基于大边际的方法相比，其泛化性能更优。然而，与其他核方法类似，ODM在计算时间和内存方面存在普遍的可扩展性问题。本文提出了一种可扩展的ODM，其训练速度相比原始ODM方法可提升近十倍。对于非线性核，我们提出了一种新颖的分布感知划分方法，使得各划分上训练的局部ODM能够接近并快速收敛至全局ODM。当采用线性核时，我们扩展了一种通信高效的SVRG方法以进一步加速训练。大量实证研究验证了所提方法具有极高的计算效率，且几乎不降低泛化性能。