Wasserstein gradient flows on probability measures have found a host of applications in various optimization problems. They typically arise as the continuum limit of exchangeable particle systems evolving by some mean-field interaction involving a gradient-type potential. However, in many problems, such as in multi-layer neural networks, the so-called particles are edge weights on large graphs whose nodes are exchangeable. Such large graphs are known to converge to continuum limits called graphons as their size grow to infinity. We show that the Euclidean gradient flow of a suitable function of the edge-weights converges to a novel continuum limit given by a curve on the space of graphons that can be appropriately described as a gradient flow or, more technically, a curve of maximal slope. Several natural functions on graphons, such as homomorphism functions and the scalar entropy, are covered by our set-up, and the examples have been worked out in detail.
翻译:概率测度上的Wasserstein梯度流已在各类优化问题中找到了众多应用。它们通常作为可交换粒子系统在涉及梯度型势的平均场相互作用下演化的连续极限出现。然而,在许多问题中(例如多层神经网络),所谓的粒子是大型图上的边权,其节点是可交换的。已知这类大型图随着规模趋于无穷,会收敛到称为图同胚的连续极限。我们证明:边权适当函数的欧几里得梯度流收敛到一种新颖的连续极限,该极限由图同胚空间上的曲线给出,可恰当地描述为梯度流,或更技术性地称为最大斜率曲线。图同胚上的若干自然函数(如同态函数和标量熵)均涵盖在我们的框架内,并已详细推导了具体实例。