When deploying machine learning solutions, they must satisfy multiple requirements beyond accuracy, such as fairness, robustness, or safety. These requirements are imposed during training either implicitly, using penalties, or explicitly, using constrained optimization methods based on Lagrangian duality. Either way, specifying requirements is hindered by the presence of compromises and limited prior knowledge about the data. Furthermore, their impact on performance can often only be evaluated by actually solving the learning problem. This paper presents a constrained learning approach that adapts the requirements while simultaneously solving the learning task. To do so, it relaxes the learning constraints in a way that contemplates how much they affect the task at hand by balancing the performance gains obtained from the relaxation against a user-defined cost of that relaxation. We call this approach resilient constrained learning after the term used to describe ecological systems that adapt to disruptions by modifying their operation. We show conditions under which this balance can be achieved and introduce a practical algorithm to compute it, for which we derive approximation and generalization guarantees. We showcase the advantages of this resilient learning method in image classification tasks involving multiple potential invariances and in heterogeneous federated learning.

翻译：在部署机器学习解决方案时，除了准确性之外，还必须满足公平性、鲁棒性或安全性等多种需求。这些需求在训练过程中通过隐式惩罚或基于拉格朗日对偶性的显式约束优化方法施加。无论采用何种方式，需求的指定都受到妥协存在和数据先验知识有限的阻碍。此外，它们对性能的影响通常只能通过实际求解学习问题来评估。本文提出一种在求解学习任务的同时自适应调整需求的约束学习方法。为此，它通过平衡松弛带来的性能提升与用户定义的松弛成本，以一种考虑松弛对当前任务影响程度的方式放宽学习约束。我们将这种方法称为弹性约束学习，借鉴了描述生态系统通过改变运作方式适应干扰的术语。我们展示了实现这种平衡的条件，并引入一种实用算法进行计算，同时推导出其近似保证和泛化保证。我们通过涉及多个潜在不变性的图像分类任务和异构联邦学习任务，展示了这种弹性学习方法的优势。