Deep neural networks are becoming increasingly popular in approximating arbitrary functions from noisy data. But wider adoption is being hindered by the need to explain such models and to impose additional constraints on them. Monotonicity constraint is one of the most requested properties in real-world scenarios and is the focus of this paper. One of the oldest ways to construct a monotonic fully connected neural network is to constrain its weights to be non-negative while employing a monotonic activation function. Unfortunately, this construction does not work with popular non-saturated activation functions such as ReLU, ELU, SELU etc, as it can only approximate convex functions. We show this shortcoming can be fixed by employing the original activation function for a part of the neurons in the layer, and employing its point reflection for the other part. Our experiments show this approach of building monotonic deep neural networks have matching or better accuracy when compared to other state-of-the-art methods such as deep lattice networks or monotonic networks obtained by heuristic regularization. This method is the simplest one in the sense of having the least number of parameters, not requiring any modifications to the learning procedure or steps post-learning steps.

翻译：深度神经网络在根据含噪数据逼近任意函数方面日益流行。然而，这类模型需要可解释性并施加额外约束，这阻碍了其更广泛的采用。单调性约束是现实场景中最常要求的属性之一，也是本文的研究重点。构建单调全连接神经网络的最古老方法之一是约束其权重为非负值，同时采用单调激活函数。遗憾的是，这种构造方法无法适用于ReLU、ELU、SELU等流行的非饱和激活函数，因为它只能逼近凸函数。我们证明，通过为层中部分神经元使用原始激活函数，为另一部分神经元使用其点反射函数，可以弥补这一缺陷。实验表明，与其他先进方法（如深度格子网络或通过启发式正则化得到的单调网络）相比，这种构建单调深度神经网络的方法具有匹配或更高的准确率。该方法在参数数量最少的意义上最为简单，无需对学习过程或学习后步骤进行任何修改。