Classification margins are commonly used to estimate the generalization ability of machine learning models. We present an empirical study of these margins in artificial neural networks. A global estimate of margin size is usually used in the literature. In this work, we point out seldom considered nuances regarding classification margins. Notably, we demonstrate that some types of training samples are modelled with consistently small margins while affecting generalization in different ways. By showing a link with the minimum distance to a different-target sample and the remoteness of samples from one another, we provide a plausible explanation for this observation. We support our findings with an analysis of fully-connected networks trained on noise-corrupted MNIST data, as well as convolutional networks trained on noise-corrupted CIFAR10 data.

翻译：分类边界通常用于估计机器学习模型的泛化能力。本文对人工神经网络中的这些边界进行了实证研究。文献中通常使用边界大小的全局估计。在本工作中，我们指出了关于分类边界鲜少被考虑的细微之处。值得注意的是，我们证明了某些类型的训练样本始终以较小的边界进行建模，同时以不同方式影响泛化性能。通过展示其与最小距离（至不同目标样本的距离）及样本之间疏离度的关联，我们为这一观察结果提供了合理的解释。我们通过在噪声污染的MNIST数据上训练的全连接网络，以及在噪声污染的CIFAR10数据上训练的卷积网络分析，佐证了我们的发现。