We develop in this paper a multi-grade deep learning method for solving nonlinear partial differential equations (PDEs). Deep neural networks (DNNs) have received super performance in solving PDEs in addition to their outstanding success in areas such as natural language processing, computer vision, and robotics. However, training a very deep network is often a challenging task. As the number of layers of a DNN increases, solving a large-scale non-convex optimization problem that results in the DNN solution of PDEs becomes more and more difficult, which may lead to a decrease rather than an increase in predictive accuracy. To overcome this challenge, we propose a two-stage multi-grade deep learning (TS-MGDL) method that breaks down the task of learning a DNN into several neural networks stacked on top of each other in a staircase-like manner. This approach allows us to mitigate the complexity of solving the non-convex optimization problem with large number of parameters and learn residual components left over from previous grades efficiently. We prove that each grade/stage of the proposed TS-MGDL method can reduce the value of the loss function and further validate this fact through numerical experiments. Although the proposed method is applicable to general PDEs, implementation in this paper focuses only on the 1D, 2D, and 3D viscous Burgers equations. Experimental results show that the proposed two-stage multi-grade deep learning method enables efficient learning of solutions of the equations and outperforms existing single-grade deep learning methods in predictive accuracy. Specifically, the predictive errors of the single-grade deep learning are larger than those of the TS-MGDL method in 26-60, 4-31 and 3-12 times, for the 1D, 2D, and 3D equations, respectively.

翻译：本文提出了一种用于求解非线性偏微分方程的多级深度学习方法。深度神经网络在自然语言处理、计算机视觉和机器人等领域取得卓越成就的同时，在求解偏微分方程方面也展现出优异性能。然而，训练非常深的网络往往是一项具有挑战性的任务。随着深度神经网络层数的增加，求解由此产生的大规模非凸优化问题（即深度神经网络对偏微分方程的求解）变得愈发困难，这可能导致预测精度下降而非提升。为克服这一挑战，我们提出了一种两阶段多级深度学习方法，该方法将深度神经网络的学习任务分解为多个以阶梯形式堆叠的神经网络。这种方法使我们能够降低求解包含大量参数的非凸优化问题的复杂度，并有效学习先前阶段残留的残差分量。我们证明了所提出的两阶段多级深度学习方法每个阶段都能降低损失函数值，并通过数值实验验证了这一事实。尽管所提出的方法适用于一般性偏微分方程，但本文的实现仅聚焦于一维、二维和三维黏性Burgers方程。实验结果表明，所提出的两阶段多级深度学习方法能够高效学习方程的解，并在预测精度上优于现有的单级深度学习方法。具体而言，对于一维、二维和三维方程，单级深度学习的预测误差分别是两阶段多级深度学习方法的26-60倍、4-31倍和3-12倍。