In this article the authors develop an intrinsic measure for quantifying heterogeneity in training data for supervised learning. This measure is the variance of a random variable which factors through the influences of pairs of training points. The variance is shown to capture data heterogeneity and can thus be used to assess if a sample is a mixture of distributions. The authors prove that the data itself contains key information that supports a partitioning into blocks. Several proof of concept studies are provided that quantify the connection between variance and heterogeneity for EMNIST image data and synthetic data. The authors establish that variance is maximal for equal mixes of distributions, and detail how variance-based data purification followed by conventional training over blocks can lead to significant increases in test accuracy.

翻译：本文中，作者提出了一种用于量化监督学习训练数据异质性的内在度量。该度量是一个随机变量的方差，该变量通过训练数据点对之间的影响进行因子分解。该方差被证明能够捕捉数据的异质性，因此可用于评估样本是否为多个分布的混合。作者证明数据本身包含支持分块划分的关键信息。研究提供了多个概念验证实验，量化了方差与异质性在EMNIST图像数据和合成数据中的关联。作者证实方差在分布均匀混合时达到最大值，并详细阐述了基于方差的数据纯化结合分块常规训练如何显著提升测试精度。