Predictive algorithms are often trained by optimizing some loss function, to which regularization functions are added to impose a penalty for violating constraints. As expected, the addition of such regularization functions can change the minimizer of the objective. It is not well-understood which regularizers change the minimizer of the loss, and, when the minimizer does change, how it changes. We use property elicitation to take first steps towards understanding the joint relationship between the loss and regularization functions and the optimal decision for a given problem instance. In particular, we give a necessary and sufficient condition on loss and regularizer pairs for when a property changes with the addition of the regularizer, and examine some regularizers satisfying this condition standard in the fair machine learning literature. We empirically demonstrate how algorithmic decision-making changes as a function of both data distribution changes and hardness of the constraints.

翻译：预测算法通常通过优化某个损失函数进行训练，并在其中加入正则化函数以对违反约束的行为施加惩罚。正如预期，这类正则化函数的引入可能会改变目标函数的极小化器。目前尚不清楚哪些正则化器会改变损失函数的极小化器，以及在极小化器发生变化时其具体如何变化。我们采用属性引出法，初步探讨损失函数与正则化函数之间的联合关系，以及针对给定问题实例的最优决策。具体而言，我们给出了当属性随正则化器加入而改变时，损失函数与正则化器对的必要充分条件，并考察了公平机器学习文献中满足该条件的若干标准正则化器。我们通过实验展示了算法决策如何随数据分布变化和约束难度变化而改变。