This paper investigates the relationships between hyperparameters of machine learning and fairness. Data-driven solutions are increasingly used in critical socio-technical applications where ensuring fairness is important. Rather than explicitly encoding decision logic via control and data structures, the ML developers provide input data, perform some pre-processing, choose ML algorithms, and tune hyperparameters (HPs) to infer a program that encodes the decision logic. Prior works report that the selection of HPs can significantly influence fairness. However, tuning HPs to find an ideal trade-off between accuracy, precision, and fairness has remained an expensive and tedious task. Can we predict fairness of HP configuration for a given dataset? Are the predictions robust to distribution shifts? We focus on group fairness notions and investigate the HP space of 5 training algorithms. We first find that tree regressors and XGBoots significantly outperformed deep neural networks and support vector machines in accurately predicting the fairness of HPs. When predicting the fairness of ML hyperparameters under temporal distribution shift, the tree regressors outperforms the other algorithms with reasonable accuracy. However, the precision depends on the ML training algorithm, dataset, and protected attributes. For example, the tree regressor model was robust for training data shift from 2014 to 2018 on logistic regression and discriminant analysis HPs with sex as the protected attribute; but not for race and other training algorithms. Our method provides a sound framework to efficiently perform fine-tuning of ML training algorithms and understand the relationships between HPs and fairness.

翻译：本文研究机器学习超参数与公平性之间的关系。数据驱动解决方案日益广泛应用于关键的社会技术应用中，确保公平性至关重要。机器学习开发者并非通过控制与数据结构显式编码决策逻辑，而是提供输入数据、执行预处理、选择机器学习算法并调整超参数，从而推导出编码决策逻辑的程序。已有研究表明超参数选择会显著影响公平性。然而，通过调整超参数在准确率、精确度与公平性之间寻求理想平衡，仍然是昂贵且繁琐的任务。我们能否预测给定数据集上超参数配置的公平性？这些预测对分布偏移是否具有鲁棒性？我们聚焦群体公平性概念，研究了5种训练算法的超参数空间。首先发现树回归器与XGBoost在准确预测超参数公平性方面显著优于深度神经网络与支持向量机。在时间分布偏移下预测机器学习超参数公平性时，树回归器以合理准确度优于其他算法。但预测精度取决于机器学习训练算法、数据集及受保护属性。例如：以性别为受保护属性时，树回归器模型对逻辑回归和判别分析超参数在2014年至2018年训练数据偏移中表现鲁棒；但对种族属性及其他训练算法则不然。本方法为高效执行机器学习训练算法微调及理解超参数与公平性关系提供了可靠框架。