Predictive algorithms are often trained by optimizing some loss function, to which regularization functions are added to impose a penalty for violating constraints. As expected, the addition of such regularization functions can change the minimizer of the objective. It is not well-understood which regularizers change the minimizer of the loss, and, when the minimizer does change, how it changes. We use property elicitation to take first steps towards understanding the joint relationship between the loss and regularization functions and the optimal decision for a given problem instance. In particular, we give a necessary and sufficient condition on loss and regularizer pairs for when a property changes with the addition of the regularizer, and examine some regularizers satisfying this condition standard in the fair machine learning literature. We empirically demonstrate how algorithmic decision-making changes as a function of both data distribution changes and hardness of the constraints.

翻译：预测算法通常通过优化某个损失函数进行训练，并在其中添加正则化函数以对违反约束的行为施加惩罚。正如预期，添加此类正则化函数会改变目标函数的最小化器。目前尚不明确哪些正则化器会改变损失函数的最小化器，以及当最小化器改变时，其变化方式如何。我们采用属性引导方法，初步探究损失函数与正则化函数之间的联合关系，以及给定问题实例中的最优决策。具体而言，我们给出了损失函数与正则化函数配对时，属性随正则化器添加而变化的充要条件，并检验了公平机器学习文献中满足该条件的若干标准正则化器。我们通过实验展示了算法决策如何随数据分布变化和约束难度变化而改变。