Directly parameterizing and learning gradients of functions has widespread significance, with specific applications in optimization, generative modeling, and optimal transport. This paper introduces gradient networks (GradNets): novel neural network architectures that parameterize gradients of various function classes. GradNets exhibit specialized architectural constraints that ensure correspondence to gradient functions. We provide a comprehensive GradNet design framework that includes methods for transforming GradNets into monotone gradient networks (mGradNets), which are guaranteed to represent gradients of convex functions. We establish the approximation capabilities of the proposed GradNet and mGradNet. Our results demonstrate that these networks universally approximate the gradients of (convex) functions. Furthermore, these networks can be customized to correspond to specific spaces of (monotone) gradient functions, including gradients of transformed sums of (convex) ridge functions. Our analysis leads to two distinct GradNet architectures, GradNet-C and GradNet-M, and we describe the corresponding monotone versions, mGradNet-C and mGradNet-M. Our empirical results show that these architectures offer efficient parameterizations and outperform popular methods in gradient field learning tasks.

翻译：直接参数化并学习函数的梯度具有广泛意义，在优化、生成建模和最优传输等领域具有特定应用。本文提出梯度网络（GradNets）：一种参数化各类函数梯度的新型神经网络架构。GradNets通过专门的架构约束确保与梯度函数的对应关系。我们提供了全面的GradNet设计框架，其中包含将GradNet转换为单调梯度网络（mGradNet）的方法——这类网络能保证表示凸函数的梯度。我们建立了所提出的GradNet和mGradNet的逼近能力，结果表明这些网络能够通用逼近（凸）函数的梯度。此外，这些网络可定制化地对应特定（单调）梯度函数空间，包括（凸）岭函数变换和的梯度。我们的分析衍生出两种不同的GradNet架构：GradNet-C和GradNet-M，并描述了对应的单调版本mGradNet-C和mGradNet-M。实验结果表明，这些架构提供高效的参数化方式，并在梯度场学习任务中优于主流方法。