This paper introduces a new neural-network-based approach, namely In-Context Operator Networks (ICON), to simultaneously learn operators from the prompted data and apply it to new questions during the inference stage, without any weight update. Existing methods are limited to using a neural network to approximate a specific equation solution or a specific operator, requiring retraining when switching to a new problem with different equations. By training a single neural network as an operator learner, we can not only get rid of retraining (even fine-tuning) the neural network for new problems, but also leverage the commonalities shared across operators so that only a few demos in the prompt are needed when learning a new operator. Our numerical results show the neural network's capability as a few-shot operator learner for a diversified type of differential equation problems, including forward and inverse problems of ordinary differential equations (ODEs), partial differential equations (PDEs), and mean-field control (MFC) problems, and also show that it can generalize its learning capability to operators beyond the training distribution.

翻译：本文提出了一种新的基于神经网络的方法，即上下文算子网络（ICON），该方法能够同时从提示数据中学习算子，并在推理阶段将其应用于新问题，而无需任何权重更新。现有方法仅限于使用神经网络逼近特定方程的解或特定算子，当切换到不同方程的新问题时需要重新训练。通过训练单个神经网络作为算子学习器，我们不仅能够避免针对新问题重新训练（甚至微调）神经网络，还能利用算子间的共性，从而在学习新算子时只需少量提示示例。我们的数值结果表明，该神经网络作为小样本算子学习器，能够处理多种类型的微分方程问题，包括常微分方程（ODEs）、偏微分方程（PDEs）的正向与逆向问题以及平均场控制（MFC）问题，并且能够将其学习能力泛化到训练分布之外的算子。