The demographic disparity of biometric systems has led to serious concerns regarding their societal impact as well as applicability of such systems in private and public domains. A quantitative evaluation of demographic fairness is an important step towards understanding, assessment, and mitigation of demographic bias in biometric applications. While few, existing fairness measures are based on post-decision data (such as verification accuracy) of biometric systems, we discuss how pre-decision data (score distributions) provide useful insights towards demographic fairness. In this paper, we introduce multiple measures, based on the statistical characteristics of score distributions, for the evaluation of demographic fairness of a generic biometric verification system. We also propose different variants for each fairness measure depending on how the contribution from constituent demographic groups needs to be combined towards the final measure. In each case, the behavior of the measure has been illustrated numerically and graphically on synthetic data. The demographic imbalance in benchmarking datasets is often overlooked during fairness assessment. We provide a novel weighing strategy to reduce the effect of such imbalance through a non-linear function of sample sizes of demographic groups. The proposed measures are independent of the biometric modality, and thus, applicable across commonly used biometric modalities (e.g., face, fingerprint, etc.).

翻译：生物识别系统的人口统计学差异引发了对其社会影响以及在私人和公共领域适用性的严重关切。定量评估人口统计学公平性是理解、评估和减轻生物识别应用中人口统计学偏倚的重要步骤。尽管现有少数公平性度量基于生物识别系统的决策后数据（如验证准确率），但我们讨论了决策前数据（分数分布）如何为人口统计学公平性提供有益见解。本文基于分数分布的统计特征，引入多种度量指标，用于评估通用生物识别验证系统的人口统计学公平性。我们还针对每种公平性度量，根据组成人口统计学群体贡献需如何组合成最终度量指标的方式，提出了不同变体。每种情况下，均通过合成数据以数值和图形方式展示了度量指标的行为。基准数据集中的群体不平衡在公平性评估中常被忽视。我们提出了一种新型加权策略，通过群体样本量的非线性函数来减少此类不平衡的影响。所提议的度量指标独立于生物识别模态，因此适用于常见生物识别模态（如人脸、指纹等）。