When deploying machine learning solutions, they must satisfy multiple requirements beyond accuracy, such as fairness, robustness, or safety. These requirements are imposed during training either implicitly, using penalties, or explicitly, using constrained optimization methods based on Lagrangian duality. Either way, specifying requirements is hindered by the presence of compromises and limited prior knowledge about the data. Furthermore, their impact on performance can often only be evaluated by actually solving the learning problem. This paper presents a constrained learning approach that adapts the requirements while simultaneously solving the learning task. To do so, it relaxes the learning constraints in a way that contemplates how much they affect the task at hand by balancing the performance gains obtained from the relaxation against a user-defined cost of that relaxation. We call this approach resilient constrained learning after the term used to describe ecological systems that adapt to disruptions by modifying their operation. We show conditions under which this balance can be achieved and introduce a practical algorithm to compute it, for which we derive approximation and generalization guarantees. We showcase the advantages of this resilient learning method in image classification tasks involving multiple potential invariances and in heterogeneous federated learning.

翻译：在部署机器学习解决方案时，它们必须满足除准确性之外的多种要求，例如公平性、鲁棒性或安全性。这些要求通过隐式惩罚或基于拉格朗日对偶性的显式约束优化方法施加于训练过程中。无论采用哪种方式，由于存在权衡和关于数据的有限先验知识，明确这些要求都会受到阻碍。此外，它们对性能的影响往往只能通过实际求解学习问题来评估。本文提出了一种约束学习方法，该方法在求解学习任务的同时自适应调整要求。具体而言，它通过放松学习约束来考虑这些约束对当前任务的影响程度，平衡放松约束带来的性能增益与用户定义的放松成本。我们将这种方法称为弹性约束学习，该术语借鉴了描述生态系统通过调整运作方式适应干扰的概念。我们展示了实现这种平衡的条件，并引入了一种实用的计算算法，推导了其近似性与泛化性保证。我们通过涉及多个潜在不变性的图像分类任务和异构联邦学习场景，展示了这种弹性学习方法的优势。