Incorporating prior knowledge or specifications of input-output relationships into machine learning models has attracted significant attention, as it enhances generalization from limited data and yields conforming outputs. However, most existing approaches use soft constraints by penalizing violations through regularization, which offers no guarantee of constraint satisfaction, especially on inputs far from the training distribution--an essential requirement in safety-critical applications. On the other hand, imposing hard constraints on neural networks may hinder their representational power, adversely affecting performance. To address this, we propose HardNet, a practical framework for constructing neural networks that inherently satisfy hard constraints without sacrificing model capacity. Unlike approaches that modify outputs only at inference time, HardNet enables end-to-end training with hard constraint guarantees, leading to improved performance. To the best of our knowledge, HardNet is the first method that enables efficient and differentiable enforcement of more than one input-dependent inequality constraint. It allows unconstrained optimization of the network parameters using standard algorithms by appending a differentiable closed-form enforcement layer to the network's output. Furthermore, we show that HardNet retains neural networks' universal approximation capabilities. We demonstrate its versatility and effectiveness across various applications: learning with piecewise constraints, learning optimization solvers with guaranteed feasibility, and optimizing control policies in safety-critical systems.

翻译：将先验知识或输入输出关系的规范融入机器学习模型已引起广泛关注，因为它能增强从有限数据中的泛化能力并产生符合要求的输出。然而，现有方法大多通过正则化惩罚违规来使用软约束，这无法保证约束满足，尤其是在远离训练分布的输入上——而这是安全关键应用中的基本要求。另一方面，对神经网络施加硬约束可能会限制其表示能力，从而对性能产生不利影响。为解决这一问题，我们提出HardNet，这是一个实用的框架，用于构建本质上满足硬约束且不牺牲模型容量的神经网络。与仅在推理时修改输出的方法不同，HardNet支持具有硬约束保证的端到端训练，从而提升性能。据我们所知，HardNet是首个能够高效且可微分地强制执行多个输入相关不等式约束的方法。它通过在网络输出后附加一个可微分的闭式强制执行层，允许使用标准算法对网络参数进行无约束优化。此外，我们证明HardNet保留了神经网络的通用逼近能力。我们在多种应用中展示了其多功能性和有效性：使用分段约束进行学习、学习具有可行性保证的优化求解器，以及在安全关键系统中优化控制策略。