The operationalization of algorithmic fairness comes with several practical challenges, not the least of which is the availability or reliability of protected attributes in datasets. In real-world contexts, practical and legal impediments may prevent the collection and use of demographic data, making it difficult to ensure algorithmic fairness. While initial fairness algorithms did not consider these limitations, recent proposals aim to achieve algorithmic fairness in classification by incorporating noisiness in protected attributes or not using protected attributes at all. To the best of our knowledge, this is the first head-to-head study of fair classification algorithms to compare attribute-reliant, noise-tolerant and attribute-blind algorithms along the dual axes of predictivity and fairness. We evaluated these algorithms via case studies on four real-world datasets and synthetic perturbations. Our study reveals that attribute-blind and noise-tolerant fair classifiers can potentially achieve similar level of performance as attribute-reliant algorithms, even when protected attributes are noisy. However, implementing them in practice requires careful nuance. Our study provides insights into the practical implications of using fair classification algorithms in scenarios where protected attributes are noisy or partially available.

翻译：算法公平性的实际应用中面临若干实践挑战，其中数据集保护属性的可用性或可靠性尤为突出。在现实场景中，实际与法律障碍可能阻碍人口统计数据的收集与使用，导致难以保障算法公平性。虽然早期公平算法未考虑这些限制，但近期研究旨在通过融入保护属性的噪声特征或完全避免使用保护属性来实现分类中的算法公平性。据我们所知，这是首个针对公平分类算法的头对头研究，在预测性与公平性双重维度上对属性依赖型、噪声容忍型与属性盲型算法进行系统比较。我们通过四个真实数据集案例研究及合成扰动评估了这些算法。研究表明，即使保护属性存在噪声，属性盲型与噪声容忍型公平分类器仍可达到与属性依赖型算法相当的性能水平。然而，实际部署需审慎把握细微差异。本研究为在保护属性存在噪声或部分可用场景中应用公平分类算法提供了实践启示。