Bias in machine learning has rightly received significant attention over the last decade. However, most fair machine learning (fair-ML) work to address bias in decision-making systems has focused solely on the offline setting. Despite the wide prevalence of online systems in the real world, work on identifying and correcting bias in the online setting is severely lacking. The unique challenges of the online environment make addressing bias more difficult than in the offline setting. First, Streaming Machine Learning (SML) algorithms must deal with the constantly evolving real-time data stream. Second, they need to adapt to changing data distributions (concept drift) to make accurate predictions on new incoming data. Adding fairness constraints to this already complicated task is not straightforward. In this work, we focus on the challenges of achieving fairness in biased data streams while accounting for the presence of concept drift, accessing one sample at a time. We present Fair Sampling over Stream ($FS^2$), a novel fair rebalancing approach capable of being integrated with SML classification algorithms. Furthermore, we devise the first unified performance-fairness metric, Fairness Bonded Utility (FBU), to evaluate and compare the trade-off between performance and fairness of different bias mitigation methods efficiently. FBU simplifies the comparison of fairness-performance trade-offs of multiple techniques through one unified and intuitive evaluation, allowing model designers to easily choose a technique. Overall, extensive evaluations show our measures surpass those of other fair online techniques previously reported in the literature.

翻译：机器学习中的偏差问题在过去十年中确实受到了广泛关注。然而，大多数旨在解决决策系统中偏差的公平机器学习（fair-ML）工作仅聚焦于离线场景。尽管在线系统在现实世界中广泛存在，但针对在线环境中偏差识别与纠正的研究却严重不足。在线环境的独特挑战使得解决偏差比离线场景更为困难。首先，流式机器学习（SML）算法必须处理持续演化的实时数据流。其次，它们需要适应不断变化的数据分布（概念漂移），以便对新到达的数据进行准确预测。在这一复杂任务中增加公平性约束并非易事。本研究聚焦于在存在概念漂移且每次仅处理一个样本的偏差数据流中实现公平性的挑战。我们提出了一种新颖的公平重采样方法——流式公平采样（$FS^2$），该方法能够与SML分类算法集成。此外，我们设计了首个统一的性能-公平性度量指标——公平绑定效用（FBU），以高效评估和比较不同偏差缓解方法在性能与公平性之间的权衡。FBU通过统一且直观的评估方式简化了多种技术中公平性-性能权衡的比较，使模型设计者能够轻松选择合适的技术。总体而言，大量实验表明，我们的度量方法超越了文献中先前报道的其他在线公平技术。