Inspired by the relation between deep neural network (DNN) and partial differential equations (PDEs), we study the general form of the PDE models of deep neural networks. To achieve this goal, we formulate DNN as an evolution operator from a simple base model. Based on several reasonable assumptions, we prove that the evolution operator is actually determined by convection-diffusion equation. This convection-diffusion equation model gives mathematical explanation for several effective networks. Moreover, we show that the convection-diffusion model improves the robustness and reduces the Rademacher complexity. Based on the convection-diffusion equation, we design a new training method for ResNets. Experiments validate the performance of the proposed method.

翻译：受深度神经网络（DNN）与偏微分方程（PDE）之间关系的启发，我们研究了深度神经网络PDE模型的一般形式。为实现这一目标，我们将DNN构建为从简单基模型出发的演化算子。基于若干合理假设，我们证明该演化算子实际上由对流-扩散方程决定。这一对流-扩散方程模型为多种有效网络提供了数学解释。此外，我们证明该对流-扩散模型能够提升鲁棒性并降低Rademacher复杂度。基于该对流-扩散方程，我们设计了ResNets的新训练方法。实验验证了所提方法的性能。