The ability to train Deep Neural Networks (DNNs) with constraints is instrumental in improving the fairness of modern machine-learning models. Many algorithms have been analysed in recent years, and yet there is no standard, widely accepted method for the constrained training of DNNs. In this paper, we provide a challenging benchmark of real-world large-scale fairness-constrained learning tasks, built on top of the US Census (Folktables). We point out the theoretical challenges of such tasks and review the main approaches in stochastic approximation algorithms. Finally, we demonstrate the use of the benchmark by implementing and comparing three recently proposed, but as-of-yet unimplemented, algorithms both in terms of optimization performance, and fairness improvement. We release the code of the benchmark as a Python package at https://github.com/humancompatible/train.

翻译：训练具有约束条件的深度神经网络（DNN）的能力对于提升现代机器学习模型的公平性至关重要。近年来，许多算法已被分析研究，但目前仍缺乏一种标准且被广泛接受的DNN约束训练方法。本文基于美国人口普查数据（Folktables），构建了一个具有挑战性的现实世界大规模公平约束学习任务基准。我们指出了此类任务在理论上面临的挑战，并回顾了随机逼近算法中的主要方法。最后，我们通过实现并比较三种近期提出但尚未被广泛实现的算法，在优化性能和公平性提升两方面展示了该基准的使用价值。我们已将基准代码以Python包形式发布于 https://github.com/humancompatible/train。