We propose derivative-informed neural operators (DINOs), a general family of neural networks to approximate operators as infinite-dimensional mappings from input function spaces to output function spaces or quantities of interest. After discretizations both inputs and outputs are high-dimensional. We aim to approximate not only the operators with improved accuracy but also their derivatives (Jacobians) with respect to the input function-valued parameter to empower derivative-based algorithms in many applications, e.g., Bayesian inverse problems, optimization under parameter uncertainty, and optimal experimental design. The major difficulties include the computational cost of generating derivative training data and the high dimensionality of the problem leading to large training cost. To address these challenges, we exploit the intrinsic low-dimensionality of the derivatives and develop algorithms for compressing derivative information and efficiently imposing it in neural operator training yielding derivative-informed neural operators. We demonstrate that these advances can significantly reduce the costs of both data generation and training for large classes of problems (e.g., nonlinear steady state parametric PDE maps), making the costs marginal or comparable to the costs without using derivatives, and in particular independent of the discretization dimension of the input and output functions. Moreover, we show that the proposed DINO achieves significantly higher accuracy than neural operators trained without derivative information, for both function approximation and derivative approximation (e.g., Gauss-Newton Hessian), especially when the training data are limited.

翻译：我们提出导数引导神经算子（DINOs），这是一类通用神经网络框架，用于近似从输入函数空间到输出函数空间或目标量的无限维映射。经过离散化后，输入和输出均为高维变量。我们旨在不仅以更高精度近似算子，还近似其对输入函数值参数的导数（雅可比矩阵），以赋能众多应用中的基于导数的算法，例如贝叶斯反问题、参数不确定性下的优化以及最优实验设计。主要挑战包括生成导数训练数据的计算成本，以及问题的高维性导致高昂的训练成本。为应对这些挑战，我们利用导数的内在低维性，开发了压缩导数信息并将其高效嵌入神经算子训练的算法，从而构建导数引导神经算子。我们证明，这些进展能够显著降低大规模问题（如非线性稳态参数偏微分方程映射）的数据生成和训练成本，使其成本趋近于或等同于未使用导数时的成本，并且尤其与输入输出函数的离散化维度无关。此外，我们表明，所提出的DINO在函数逼近和导数逼近（如高斯-牛顿海森矩阵）方面，均比未使用导数信息训练的神经算子实现显著更高的精度，尤其在训练数据有限时。