Machine learning systems often acquire biases by leveraging undesired features in the data, impacting accuracy variably across different sub-populations. Current understanding of bias formation mostly focuses on the initial and final stages of learning, leaving a gap in knowledge regarding the transient dynamics. To address this gap, this paper explores the evolution of bias in a teacher-student setup modeling different data sub-populations with a Gaussian-mixture model. We provide an analytical description of the stochastic gradient descent dynamics of a linear classifier in this setting, which we prove to be exact in high dimension. Notably, our analysis reveals how different properties of sub-populations influence bias at different timescales, showing a shifting preference of the classifier during training. Applying our findings to fairness and robustness, we delineate how and when heterogeneous data and spurious features can generate and amplify bias. We empirically validate our results in more complex scenarios by training deeper networks on synthetic and real datasets, including CIFAR10, MNIST, and CelebA.

翻译：机器学习系统常通过利用数据中不期望的特征而习得偏差，从而在不同子群体中以不同方式影响准确性。当前对偏差形成的理解主要集中于学习的初始和最终阶段，对瞬态动态过程的认识仍存在空白。为填补这一空白，本文通过高斯混合模型建模不同数据子群体，在师生框架下探讨偏差的演化过程。我们给出了该设定中线性分类器随机梯度下降动态的解析描述，并证明该描述在高维情形下是精确的。值得注意的是，我们的分析揭示了子群体的不同特性如何在不同时间尺度上影响偏差，展现了分类器在训练过程中偏好度的动态迁移。将研究结果应用于公平性与鲁棒性领域，我们明确了异构数据和伪特征在何时以及如何产生并放大偏差。通过在合成数据集和真实数据集（包括CIFAR10、MNIST和CelebA）上训练更深层网络，我们在更复杂场景中对理论结果进行了实证验证。