Research in machine learning has polarized into two general approaches for regression tasks: Transductive methods construct estimates directly from available data but are usually problem unspecific. Inductive methods can be much more specific but generally require compute-intensive solution searches. In this work, we propose a hybrid approach and show that transductive regression principles can be meta-learned through gradient descent to form efficient in-context neural approximators by leveraging the theory of vector-valued Reproducing Kernel Banach Spaces (RKBS). We apply this approach to function spaces defined over finite and infinite-dimensional spaces (function-valued operators) and show that once trained, the Transducer can almost instantaneously capture an infinity of functional relationships given a few pairs of input and output examples and return new image estimates. We demonstrate the benefit of our meta-learned transductive approach to model complex physical systems influenced by varying external factors with little data at a fraction of the usual deep learning training computational cost for partial differential equations and climate modeling applications.

翻译：机器学习研究在回归任务上已分化为两种通用方法：传导方法直接从可用数据构建估计，但通常缺乏问题特异性；归纳方法可具备高度特异性，但通常需要计算密集型的解搜索。在本工作中，我们提出一种混合方法，并证明通过利用向量值再生核巴拿赫空间（RKBS）理论，传导回归原理可通过梯度下降进行元学习，形成高效的上下文神经逼近器。我们将该方法应用于定义在有限维和无限维空间上的函数空间（函数值算子），并展示经训练后，传导器可在给定少量输入输出示例对的情况下，几乎瞬时捕捉无穷多的函数关系并返回新图像估计。我们证明了这种元学习传导方法在模拟受多种外部因素影响的复杂物理系统时的优势——在偏微分方程和气候建模应用中，仅需少量数据，计算成本仅为常规深度学习训练的一小部分。