This paper introduces the sigma flow model for the prediction of structured labelings of data observed on Riemannian manifolds, including Euclidean image domains as special case. The approach combines the Laplace-Beltrami framework for image denoising and enhancement, introduced by Sochen, Kimmel and Malladi about 25 years ago, and the assignment flow approach introduced and studied by the authors. The sigma flow arises as Riemannian gradient flow of generalized harmonic energies and thus is governed by a nonlinear geometric PDE which determines a harmonic map from a closed Riemannian domain manifold to a statistical manifold, equipped with the Fisher-Rao metric from information geometry. A specific ingredient of the sigma flow is the mutual dependency of the Riemannian metric of the domain manifold on the evolving state. This makes the approach amenable to machine learning in a specific way, by realizing this dependency through a mapping with compact time-variant parametrization that can be learned from data. Proof of concept experiments demonstrate the expressivity of the sigma flow model and prediction performance. Structural similarities to transformer network architectures and networks generated by the geometric integration of sigma flows are pointed out, which highlights the connection to deep learning and, conversely, may stimulate the use of geometric design principles for structured prediction in other areas of scientific machine learning.

翻译：本文提出了一种sigma流模型，用于预测黎曼流形上观测数据的结构化标注，其中欧几里得图像域作为特例。该方法融合了Sochen、Kimmel和Malladi约25年前提出的拉普拉斯-贝尔特拉米图像去噪与增强框架，以及作者提出并研究的分配流方法。sigma流作为广义调和能量的黎曼梯度流而产生，因此由非线性几何偏微分方程所控制，该方程确定了一个从闭黎曼定义域流形到统计流形的调和映射，其中统计流形配备了来自信息几何的Fisher-Rao度量。sigma流的一个关键特征是其定义域流形的黎曼度量对演化状态的相互依赖性。通过利用具有紧凑时变参数化的映射来实现这种依赖性，并可从数据中学习该映射，这使得该方法能够以特定方式适用于机器学习。概念验证实验展示了sigma流模型的表达能力和预测性能。研究指出了该模型与Transformer网络架构以及通过sigma流几何积分生成的网络之间的结构相似性，这凸显了其与深度学习的联系；反之，也可能促进几何设计原则在科学机器学习其他领域的结构化预测中的应用。