We investigate deep morphological neural networks (DMNNs). We demonstrate that despite their inherent non-linearity, "linear" activations are essential for DMNNs. To preserve their inherent sparsity, we propose architectures that constraint the parameters of the "linear" activations: For the first (resp. second) architecture, we work under the constraint that the majority of parameters (resp. learnable parameters) should be part of morphological operations. We improve the generalization ability of our networks via residual connections and weight dropout. Our proposed networks can be successfully trained, and are more prunable than linear networks. To the best of our knowledge, we are the first to successfully train DMNNs under such constraints. Finally, we propose a hybrid network architecture combining linear and morphological layers, showing empirically that the inclusion of morphological layers significantly accelerates the convergence of gradient descent with large batches.

翻译：本文研究深度形态学神经网络（DMNNs）。我们证明，尽管DMNNs具有固有的非线性特性，但“线性”激活函数对其至关重要。为保持其固有的稀疏性，我们提出了约束“线性”激活函数参数的架构：对于第一种（相应地，第二种）架构，我们要求在多数参数（相应地，可学习参数）应属于形态学运算的约束条件下进行设计。通过残差连接和权重丢弃技术，我们提升了网络的泛化能力。所提出的网络能够被成功训练，且比线性网络具有更强的可剪枝性。据我们所知，我们是首个在此类约束下成功训练DMNNs的研究。最后，我们提出了一种结合线性层与形态学层的混合网络架构，并通过实验证明形态学层的加入能显著加速梯度下降在大批量训练时的收敛速度。