Fairness-aware machine learning has garnered significant attention in recent years because of extensive use of machine learning in sensitive applications like judiciary systems. Various heuristics, and optimization frameworks have been proposed to enforce fairness in classification \cite{del2020review} where the later approaches either provides empirical results or provides fairness guarantee for the exact minimizer of the objective function \cite{celis2019classification}. In modern machine learning, Stochastic Gradient Descent (SGD) type algorithms are almost always used as training algorithms implying that the learned model, and consequently, its fairness properties are random. Hence, especially for crucial applications, it is imperative to construct Confidence Interval (CI) for the fairness of the learned model. In this work we provide CI for test unfairness when a group-fairness-aware, specifically, Disparate Impact (DI), and Disparate Mistreatment (DM) aware linear binary classifier is trained using online SGD-type algorithms. We show that asymptotically a Central Limit Theorem holds for the estimated model parameter of both DI and DM-aware models. We provide online multiplier bootstrap method to estimate the asymptotic covariance to construct online CI. To do so, we extend the known theoretical guarantees shown on the consistency of the online bootstrap method for unconstrained SGD to constrained optimization which could be of independent interest. We illustrate our results on synthetic and real datasets.

翻译：公平感知机器学习近年来因机器学习在司法系统等敏感应用中的广泛使用而备受关注。已有多种启发式方法和优化框架被提出用于在分类任务中强制执行公平性\cite{del2020review}，其中后者或提供经验结果，或为目标函数的精确极小化器提供公平性保证\cite{celis2019classification}。在现代机器学习中，随机梯度下降（SGD）类算法几乎始终被用作训练算法，这意味着学习到的模型及其公平性属性具有随机性。因此，尤其在关键应用中，构建学习模型公平性的置信区间（CI）至关重要。在本研究中，我们针对使用在线SGD类算法训练的、具有群体公平性意识的线性二分类器（具体涉及差异性影响（DI）和差异性对待（DM）感知），提供了测试不公平性的置信区间。我们证明，对于DI和DM感知模型，估计模型参数渐近地满足中心极限定理。我们利用在线乘子自助法来估计渐近协方差，以构建在线置信区间。为此，我们将已知的、针对无约束SGD的在线自助法一致性的理论保证扩展至约束优化问题，这一扩展本身可能具有独立的研究价值。我们在合成数据集和真实数据集上展示了我们的结果。