This paper presents RAYEN, a framework to impose hard convex constraints on the output or latent variable of a neural network. RAYEN guarantees that, for any input or any weights of the network, the constraints are satisfied at all times. Compared to other approaches, RAYEN does not perform a computationally-expensive orthogonal projection step onto the feasible set, does not rely on soft constraints (which do not guarantee the satisfaction of the constraints at test time), does not use conservative approximations of the feasible set, and does not perform a potentially slow inner gradient descent correction to enforce the constraints. RAYEN supports any combination of linear, convex quadratic, second-order cone (SOC), and linear matrix inequality (LMI) constraints, achieving a very small computational overhead compared to unconstrained networks. For example, it is able to impose 1K quadratic constraints on a 1K-dimensional variable with an overhead of less than 8 ms, and an LMI constraint with 300x300 dense matrices on a 10K-dimensional variable in less than 12 ms. When used in neural networks that approximate the solution of constrained optimization problems, RAYEN achieves computation times between 20 and 7468 times faster than state-of-the-art algorithms, while guaranteeing the satisfaction of the constraints at all times and obtaining a cost very close to the optimal one.

翻译：摘要：本文提出RAYEN框架，用于对神经网络的输出或隐变量施加硬凸约束。RAYEN保证，对于任意输入或网络权重，约束条件始终得到满足。与其他方法相比，RAYEN无需在可行集上进行计算代价高昂的正交投影步骤，不依赖软约束（后者无法保证测试时约束的满足性），不使用可行集的保守近似，也不需进行可能缓慢的内梯度下降校正来强制执行约束。RAYEN支持线性约束、凸二次约束、二阶锥（SOC）约束和线性矩阵不等式（LMI）约束的任意组合，相比无约束网络仅产生极小的计算开销。例如，它能对1000维变量施加1000个二次约束，开销不到8毫秒；对10000维变量施加包含300×300稠密矩阵的LMI约束，耗时不到12毫秒。当用于近似求解约束优化问题的神经网络时，RAYEN的计算速度比最先进算法快20至7468倍，同时始终保证约束满足性，且所得代价非常接近最优值。