Incorporating prior knowledge or specifications of input-output relationships into machine learning models has gained significant attention, as it enhances generalization from limited data and leads to conforming outputs. However, most existing approaches use soft constraints by penalizing violations through regularization, which offers no guarantee of constraint satisfaction -- an essential requirement in safety-critical applications. On the other hand, imposing hard constraints on neural networks may hinder their representational power, adversely affecting performance. To address this, we propose HardNet, a practical framework for constructing neural networks that inherently satisfy hard constraints without sacrificing model capacity. Specifically, we encode affine and convex hard constraints, dependent on both inputs and outputs, by appending a differentiable projection layer to the network's output. This architecture allows unconstrained optimization of the network parameters using standard algorithms while ensuring constraint satisfaction by construction. Furthermore, we show that HardNet retains the universal approximation capabilities of neural networks. We demonstrate the versatility and effectiveness of HardNet across various applications: fitting functions under constraints, learning optimization solvers, optimizing control policies in safety-critical systems, and learning safe decision logic for aircraft systems.

翻译：将先验知识或输入输出关系的规范融入机器学习模型已引起广泛关注，因为这能增强模型在有限数据下的泛化能力并确保输出符合要求。然而，现有方法大多通过正则化惩罚违规来实现软约束，无法保证约束的严格满足——这在安全关键型应用中是不可或缺的。另一方面，对神经网络施加硬约束可能会削弱其表示能力，从而对性能产生不利影响。为解决这一问题，我们提出HardNet，一种实用的神经网络构建框架，该框架能在不牺牲模型容量的前提下天然满足硬约束。具体而言，我们通过在网络输出端添加可微投影层，实现对输入输出相关的仿射与凸硬约束的编码。这种架构允许使用标准算法对网络参数进行无约束优化，同时通过构造确保约束满足。此外，我们证明了HardNet保持了神经网络的通用逼近能力。我们在多个应用中展示了HardNet的通用性与有效性：约束下的函数拟合、优化求解器学习、安全关键系统中的控制策略优化，以及航空器系统的安全决策逻辑学习。