Deep neural network is a powerful tool for many tasks. Understanding why it is so successful and providing a mathematical explanation is an important problem and has been one popular research direction in past years. In the literature of mathematical analysis of deep deep neural networks, a lot of works are dedicated to establishing representation theories. How to make connections between deep neural networks and mathematical algorithms is still under development. In this paper, we give an algorithmic explanation for deep neural networks, especially in their connection with operator splitting and multigrid methods. We show that with certain splitting strategies, operator-splitting methods have the same structure as networks. Utilizing this connection and the Potts model for image segmentation, two networks inspired by operator-splitting methods are proposed. The two networks are essentially two operator-splitting algorithms solving the Potts model. Numerical experiments are presented to demonstrate the effectiveness of the proposed networks.
翻译:深度神经网络是处理众多任务的有力工具。理解其成功的原因并提供数学解释是一个重要问题,也是过去数年里广受关注的研究方向之一。在深度神经网络的数学分析文献中,大量工作致力于建立表示理论。然而,如何建立深度神经网络与数学算法之间的联系仍有待发展。本文从算法角度对深度神经网络进行解释,尤其关注其与算子分裂方法和多重网格方法的关联。我们证明,采用特定分裂策略时,算子分裂方法与网络具有相同的结构。利用这一联系以及用于图像分割的Potts模型,本文提出了两种受算子分裂方法启发的网络。这两种网络本质上是求解Potts模型的两种算子分裂算法。数值实验展示了所提出网络的有效性。