In real world datasets, particular groups are under-represented, much rarer than others, and machine learning classifiers will often preform worse on under-represented populations. This problem is aggravated across many domains where datasets are class imbalanced, with a minority class far rarer than the majority class. Naive approaches to handle under-representation and class imbalance include training sub-population specific classifiers that handle class imbalance or training a global classifier that overlooks sub-population disparities and aims to achieve high overall accuracy by handling class imbalance. In this study, we find that these approaches are vulnerable in class imbalanced datasets with minority sub-populations. We introduced Fair-Net, a branched multitask neural network architecture that improves both classification accuracy and probability calibration across identifiable sub-populations in class imbalanced datasets. Fair-Nets is a straightforward extension to the output layer and error function of a network, so can be incorporated in far more complex architectures. Empirical studies with three real world benchmark datasets demonstrate that Fair-Net improves classification and calibration performance, substantially reducing performance disparity between gender and racial sub-populations.

翻译：在现实世界的数据集中，特定群体往往代表性不足，其数量远低于其他群体，机器学习分类器在这些代表性不足的群体上通常表现较差。这一问题在许多领域中因数据集类别不平衡而加剧，其中少数类别的样本量远少于多数类别。处理代表性不足和类别不平衡的朴素方法包括：训练针对子群体特定分类器以处理类别不平衡，或者训练一个忽略子群体差异的全局分类器，旨在通过处理类别不平衡实现高整体准确率。本研究发现，这些方法在面对包含少数子群体的类别不平衡数据集时存在脆弱性。我们引入了公平网络（Fair-Net），这是一种分支多任务神经网络架构，能够改善类别不平衡数据集中可识别子群体的分类准确率和概率校准性能。公平网络是对网络输出层和误差函数的直接扩展，因此可被集成到更复杂的架构中。基于三个真实世界基准数据集的实证研究表明，公平网络提升了分类与校准性能，显著减少了性别和种族子群体之间的性能差异。