When deploying machine learning solutions, they must satisfy multiple requirements beyond accuracy, such as fairness, robustness, or safety. These requirements are imposed during training either implicitly, using penalties, or explicitly, using constrained optimization methods based on Lagrangian duality. Either way, specifying requirements is hindered by the presence of compromises and limited prior knowledge about the data. Furthermore, their impact on performance can often only be evaluated by actually solving the learning problem. This paper presents a constrained learning approach that adapts the requirements while simultaneously solving the learning task. To do so, it relaxes the learning constraints in a way that contemplates how much they affect the task at hand by balancing the performance gains obtained from the relaxation against a user-defined cost of that relaxation. We call this approach resilient constrained learning after the term used to describe ecological systems that adapt to disruptions by modifying their operation. We show conditions under which this balance can be achieved and introduce a practical algorithm to compute it, for which we derive approximation and generalization guarantees. We showcase the advantages of this resilient learning method in image classification tasks involving multiple potential invariances and in heterogeneous federated learning.

翻译：在部署机器学习解决方案时，除准确性外还需满足公平性、鲁棒性或安全性等多重需求。这些需求在训练阶段通过隐式惩罚或显式基于拉格朗日对偶的约束优化方法施加。然而，由于存在折中关系且对数据的先验知识有限，明确设定需求面临困难。此外，需求对性能的影响往往只有实际求解学习问题后才能评估。本文提出一种能够在求解学习任务的同时自适应调整需求的约束学习方法。该方法通过放松学习约束，权衡放松带来的性能提升与用户定义的放松代价，从而考量约束对当前任务的影响程度。我们借鉴描述生态系统通过调整运作适应干扰的术语，将这一方法称为弹性约束学习。我们证明了实现这种平衡的条件，并引入一种实用算法进行计算，同时给出了该算法的近似性与泛化性保证。通过在涉及多种潜在不变性的图像分类任务以及异构联邦学习中，我们展示了这种弹性学习方法的优势。