Research in Machine Learning has polarized into two general regression approaches: Transductive methods derive estimates directly from available data but are usually problem unspecific. Inductive methods can be much more particular, but generally require tuning and compute-intensive searches for solutions. In this work, we adopt a hybrid approach: We leverage the theory of Reproducing Kernel Banach Spaces (RKBS) and show that transductive principles can be induced through gradient descent to form efficient \textit{in-context} neural approximators. We apply this approach to RKBS of function-valued operators and show that once trained, our \textit{Transducer} model can capture on-the-fly relationships between infinite-dimensional input and output functions, given a few example pairs, and return new function estimates. We demonstrate the benefit of our transductive approach to model complex physical systems influenced by varying external factors with little data at a fraction of the usual deep learning training computation cost for partial differential equations and climate modeling applications.

翻译：机器学习研究已分化为两种通用的回归方法：转导方法直接根据可用数据推导估计，但通常缺乏问题特异性；归纳方法可更具针对性，但一般需要调参和计算密集的搜索来寻找解。在这项工作中，我们采用混合方法：利用再生核巴拿赫空间（RKBS）理论，并证明转导原则可通过梯度下降被诱导形成高效的\textit{上下文内}神经逼近器。我们将此方法应用于函数值算子的RKBS，并表明经过训练后，我们的\textit{Transducer}模型能在给定少量示例对的情况下，捕捉无限维输入与输出函数之间的即时关系，并返回新的函数估计。我们展示了转导方法在受不同外部因素影响的复杂物理系统建模中的优势，仅需少量数据，且计算成本远低于偏微分方程和气候建模应用中常规的深度学习训练。