When deploying machine learning solutions, they must satisfy multiple requirements beyond accuracy, such as fairness, robustness, or safety. These requirements are imposed during training either implicitly, using penalties, or explicitly, using constrained optimization methods based on Lagrangian duality. Either way, specifying requirements is hindered by the presence of compromises and limited prior knowledge about the data. Furthermore, their impact on performance can often only be evaluated by actually solving the learning problem. This paper presents a constrained learning approach that adapts the requirements while simultaneously solving the learning task. To do so, it relaxes the learning constraints in a way that contemplates how much they affect the task at hand by balancing the performance gains obtained from the relaxation against a user-defined cost of that relaxation. We call this approach resilient constrained learning after the term used to describe ecological systems that adapt to disruptions by modifying their operation. We show conditions under which this balance can be achieved and introduce a practical algorithm to compute it, for which we derive approximation and generalization guarantees. We showcase the advantages of this resilient learning method in image classification tasks involving multiple potential invariances and in heterogeneous federated learning.

翻译：在部署机器学习解决方案时，除了准确性之外，还必须满足公平性、鲁棒性或安全性等多重需求。这些需求在训练过程中通过隐性惩罚或基于拉格朗日对偶性的显式约束优化方法施加。无论采用何种方式，需求设定都因存在权衡以及数据先验知识的局限性而受到阻碍。此外，这些需求对性能的影响往往只能通过实际求解学习问题来评估。本文提出一种约束学习方法，该方法在求解学习任务的同时自适应调整需求。具体而言，它通过放松学习约束，并权衡放松带来的性能增益与用户定义的放松代价，以考虑约束对当前任务的影响程度。我们将这种方法称为弹性约束学习，借用描述生态系统的术语，即通过调整自身运作来适应干扰的系统。我们展示了实现这种平衡的条件，并引入了一种实用的计算算法，同时给出了该算法的近似保证和泛化保证。我们通过涉及多种潜在不变性的图像分类任务和异构联邦学习场景，展示了这种弹性学习方法的优势。