With the emergence of powerful representations of continuous data in the form of neural fields, there is a need for discretization invariant learning: an approach for learning maps between functions on continuous domains without being sensitive to how the function is sampled. We present a new framework for understanding and designing discretization invariant neural networks (DI-Nets), which generalizes many discrete networks such as convolutional neural networks as well as continuous networks such as neural operators. Our analysis establishes upper bounds on the deviation in model outputs under different finite discretizations, and highlights the central role of point set discrepancy in characterizing such bounds. This insight leads to the design of a family of neural networks driven by numerical integration via quasi-Monte Carlo sampling with discretizations of low discrepancy. We prove by construction that DI-Nets universally approximate a large class of maps between integrable function spaces, and show that discretization invariance also describes backpropagation through such models. Applied to neural fields, convolutional DI-Nets can learn to classify and segment visual data under various discretizations, and sometimes generalize to new types of discretizations at test time. Code: https://github.com/clintonjwang/DI-net.

翻译：随着神经场作为连续数据强大表示形式的涌现，亟需离散不变学习方法——一种在连续域上学习函数间映射时不受采样方式影响的框架。我们提出了一种理解与设计离散不变神经网络（DI-Nets）的新框架，该框架将卷积神经网络等离散网络与神经算子等连续网络统一泛化。我们的分析建立了不同有限离散化下模型输出偏差的上界，并揭示了点集差异在刻画此类上界中的核心作用。这一洞见驱动了基于低差异离散化拟蒙特卡洛数值积分方法的一族神经网络设计。我们通过构造性证明表明，DI-Nets可通用逼近可积函数空间间的大类映射，并证明离散不变性同样描述了此类模型的反向传播过程。应用于神经场时，卷积DI-Nets能在不同离散化下学习视觉数据的分类与分割任务，有时甚至可在测试时泛化至新型离散化。代码：https://github.com/clintonjwang/DI-net。