We present polynomial-augmented neural networks (PANNs), a novel machine learning architecture that combines deep neural networks (DNNs) with a polynomial approximant. PANNs combine the strengths of DNNs (flexibility and efficiency in higher-dimensional approximation) with those of polynomial approximation (rapid convergence rates for smooth functions). To aid in both stable training and enhanced accuracy over a variety of problems, we present (1) a family of orthogonality constraints that impose mutual orthogonality between the polynomial and the DNN within a PANN; (2) a simple basis pruning approach to combat the curse of dimensionality introduced by the polynomial component; and (3) an adaptation of a polynomial preconditioning strategy to both DNNs and polynomials. We test the resulting architecture for its polynomial reproduction properties, ability to approximate both smooth functions and functions of limited smoothness, and as a method for the solution of partial differential equations (PDEs). Through these experiments, we demonstrate that PANNs offer superior approximation properties to DNNs for both regression and the numerical solution of PDEs, while also offering enhanced accuracy over both polynomial and DNN-based regression (each) when regressing functions with limited smoothness.

翻译：本文提出多项式增强神经网络（PANNs）——一种将深度神经网络（DNNs）与多项式逼近器相结合的新型机器学习架构。PANNs融合了深度神经网络（在高维近似中具有灵活性与高效性）和多项式近似（对光滑函数具有快速收敛速度）的双重优势。为提升模型在不同问题中的训练稳定性与精度，我们提出：（1）一系列正交性约束，强制PANN内部多项式与深度神经网络分量相互正交；（2）一种简易的基函数剪枝方法，以缓解多项式分量引发的维度灾难问题；（3）适用于深度神经网络与多项式分量的多项式预处理策略适配方案。我们通过多项实验验证了该架构的多项式重构特性、对光滑函数与有限光滑度函数的逼近能力，以及作为偏微分方程（PDEs）求解方法的有效性。实验结果表明：在回归任务与偏微分方程数值求解中，PANNs相比纯深度神经网络具有更优越的近似性能；对于有限光滑度函数的回归问题，其精度亦超越单纯基于多项式或深度神经网络的回归方法。