Deep Neural Networks (DNNs) outshine alternative function approximators in many settings thanks to their modularity in composing any desired differentiable operator. The formed parametrized functional is then tuned to solve a task at hand from simple gradient descent. This modularity comes at the cost of making strict enforcement of constraints on DNNs, e.g. from a priori knowledge of the task, or from desired physical properties, an open challenge. In this paper we propose the first provable affine constraint enforcement method for DNNs that only requires minimal changes into a given DNN's forward-pass, that is computationally friendly, and that leaves the optimization of the DNN's parameter to be unconstrained, i.e. standard gradient-based method can be employed. Our method does not require any sampling and provably ensures that the DNN fulfills the affine constraint on a given input space's region at any point during training, and testing. We coin this method POLICE, standing for Provably Optimal LInear Constraint Enforcement. Github: https://github.com/RandallBalestriero/POLICE

翻译：深度神经网络（DNN）因其在组合任意所需可微算子方面的模块性，在许多场景中优于其他函数逼近器。形成的参数化函数随后通过简单的梯度下降进行调整，以解决当前任务。这种模块性以严格强制执行DNN约束（例如来自任务的先验知识或期望的物理属性）为代价，这仍是一个开放性挑战。在本文中，我们提出了首个可证明的DNN仿射约束强制执行方法，该方法仅需对给定DNN的前向传播进行最小改动，计算友好，且使DNN参数的优化保持无约束，即可采用标准梯度方法。我们的方法无需任何采样，并可证明在训练和测试的任何时刻，DNN在给定输入空间区域上满足仿射约束。我们称此方法为POLICE，即可证明最优线性约束强制执行（Provably Optimal LInear Constraint Enforcement）。GitHub: https://github.com/RandallBalestriero/POLICE