Deep neural networks are becoming increasingly popular in approximating arbitrary functions from noisy data. But wider adoption is being hindered by the need to explain such models and to impose additional constraints on them. Monotonicity constraint is one of the most requested properties in real-world scenarios and is the focus of this paper. One of the oldest ways to construct a monotonic fully connected neural network is to constrain its weights to be non-negative while employing a monotonic activation function. Unfortunately, this construction does not work with popular non-saturated activation functions such as ReLU, ELU, SELU etc, as it can only approximate convex functions. We show this shortcoming can be fixed by employing the original activation function for a part of the neurons in the layer, and employing its point reflection for the other part. Our experiments show this approach of building monotonic deep neural networks have matching or better accuracy when compared to other state-of-the-art methods such as deep lattice networks or monotonic networks obtained by heuristic regularization. This method is the simplest one in the sense of having the least number of parameters, not requiring any modifications to the learning procedure or steps post-learning steps.

翻译：深度神经网络在从含噪数据中逼近任意函数方面日益流行。然而，这类模型的可解释性需求以及对它们施加额外约束的要求，正阻碍其更广泛的应用。单调性约束是现实场景中最常要求的性质之一，也是本文的研究重点。构建单调全连接神经网络的最经典方法之一，是约束其权重为非负，同时采用单调激活函数。不幸的是，这种构造方式无法适用于ReLU、ELU、SELU等流行的非饱和激活函数，因为它只能逼近凸函数。我们证明，这一缺陷可以通过对层内一部分神经元使用原始激活函数，而对另一部分神经元使用其逐点反射来修正。实验表明，与其他最先进的方法（如深度格点网络或通过启发式正则化获得的单调网络）相比，这种构建单调深度神经网络的方法具有匹配或更高的精度。从参数数量最少、无需修改学习过程或学习后步骤的意义上说，该方法是最简单的。