We propose derivative-informed neural operators (DINOs), a general family of neural networks to approximate operators as infinite-dimensional mappings from input function spaces to output function spaces or quantities of interest. After discretizations both inputs and outputs are high-dimensional. We aim to approximate not only the operators with improved accuracy but also their derivatives (Jacobians) with respect to the input function-valued parameter to empower derivative-based algorithms in many applications, e.g., Bayesian inverse problems, optimization under parameter uncertainty, and optimal experimental design. The major difficulties include the computational cost of generating derivative training data and the high dimensionality of the problem leading to large training cost. To address these challenges, we exploit the intrinsic low-dimensionality of the derivatives and develop algorithms for compressing derivative information and efficiently imposing it in neural operator training yielding derivative-informed neural operators. We demonstrate that these advances can significantly reduce the costs of both data generation and training for large classes of problems (e.g., nonlinear steady state parametric PDE maps), making the costs marginal or comparable to the costs without using derivatives, and in particular independent of the discretization dimension of the input and output functions. Moreover, we show that the proposed DINO achieves significantly higher accuracy than neural operators trained without derivative information, for both function approximation and derivative approximation (e.g., Gauss-Newton Hessian), especially when the training data are limited.

翻译：[translated abstract in Chinese] 我们提出导数引导神经算子（DINOs），这是一类通用的神经网络框架，用于将算子近似为从输入函数空间到输出函数空间或目标量的无限维映射。经过离散化后，输入和输出均为高维。我们的目标不仅是更准确地近似算子本身，还包括其对输入函数值参数的导数（雅可比矩阵），以支持众多应用中基于导数的算法，例如贝叶斯反问题、参数不确定性下的优化以及最优实验设计。主要难点包括生成导数训练数据的计算成本，以及问题的高维性导致训练成本高昂。为应对这些挑战，我们利用导数的内在低维特性，开发了压缩导数信息并高效将其引入神经算子训练中的算法，从而生成导数引导神经算子。我们证明，这些进展能显著降低大规模问题（如非线性稳态参数化偏微分方程映射）的数据生成和训练成本，使其边际化或与不使用导数时的成本相当，尤其不依赖于输入与输出函数的离散化维度。此外，我们表明，所提出的DINO在函数逼近和导数逼近（如高斯-牛顿海森矩阵）方面，均比未使用导数信息训练的神经算子获得显著更高的精度，尤其是在训练数据有限的情况下。