Training and running deep neural networks (NNs) often demands a lot of computation and energy-intensive specialized hardware (e.g. GPU, TPU...). One way to reduce the computation and power cost is to use binary weight NNs, but these are hard to train because the sign function has a non-smooth gradient. We present a model based on Mathematical Morphology (MM), which can binarize ConvNets without losing performance under certain conditions, but these conditions may not be easy to satisfy in real-world scenarios. To solve this, we propose two new approximation methods and develop a robust theoretical framework for ConvNets binarization using MM. We propose as well regularization losses to improve the optimization. We empirically show that our model can learn a complex morphological network, and explore its performance on a classification task.

翻译：训练和运行深度神经网络（NN）通常需要大量的计算资源和能源密集型专用硬件（例如 GPU、TPU 等）。降低计算和功耗成本的一种方法是使用二值权重神经网络，但由于符号函数具有非平滑梯度，这类网络难以训练。我们提出了一种基于数学形态学（MM）的模型，该模型在特定条件下能够在不损失性能的情况下对卷积神经网络（ConvNets）进行二值化，但这些条件在现实场景中可能难以满足。为解决这一问题，我们提出了两种新的近似方法，并构建了基于形态学的ConvNets二值化的稳健理论框架。我们还提出了正则化损失以改善优化过程。我们通过实验证明，该模型能够学习复杂的形态学网络，并探究了其在分类任务上的性能。