Machine Learning (ML) has deeply changed some fields recently, like Language and Vision and we may expect it to be relevant also to the analysis of of complex systems. Here we want to tackle the question of how and to which extent can one regress scale-free processes, i.e. processes displaying power law behavior, like earthquakes or avalanches? We are interested in predicting the large ones, i.e. rare events in the training set which therefore require extrapolation capabilities of the model. For this we consider two paradigmatic problems that are statistically self-similar. The first one is a 2-dimensional fractional Gaussian field obeying linear dynamics, self-similar by construction and amenable to exact analysis. The second one is the Abelian sandpile model, exhibiting self-organized criticality. The emerging paradigm of Geometric Deep Learning shows that including known symmetries into the model's architecture is key to success. Here one may hope to extrapolate only by leveraging scale invariance. This is however a peculiar symmetry, as it involves possibly non-trivial coarse-graining operations and anomalous scaling. We perform experiments on various existing architectures like U-net, Riesz network (scale invariant by construction), or our own proposals: a wavelet-decomposition based Graph Neural Network (with discrete scale symmetry), a Fourier embedding layer and a Fourier-Mellin Neural Operator. Based on these experiments and a complete characterization of the linear case, we identify the main issues relative to spectral biases and coarse-grained representations, and discuss how to alleviate them with the relevant inductive biases.

翻译：机器学习（ML）近年来已深刻改变了语言与视觉等领域，我们亦可预期其在复杂系统分析中具有重要价值。本文旨在探讨如何以及在何种程度上能够回归无标度过程，即呈现幂律行为的过程（如地震或雪崩）。我们关注对大规模事件（即训练集中罕见事件）的预测，这要求模型具备外推能力。为此，我们研究了两个统计自相似的典型问题：其一是服从线性动力学的二维分数高斯场，该模型通过构造具有自相似性且可进行精确分析；其二是展现自组织临界性的阿贝尔沙堆模型。几何深度学习的新范式表明，将已知对称性纳入模型架构是成功的关键。此处我们期望仅通过利用尺度不变性实现外推。然而这是一种特殊的对称性，因其可能涉及非平凡的粗粒度操作与反常标度行为。我们在多种现有架构（如U-net、构造上尺度不变的Riesz网络）及我们提出的新架构（包括基于小波分解的图神经网络（具有离散尺度对称性）、傅里叶嵌入层和傅里叶-梅林神经算子）上进行了实验。基于这些实验及对线性情况的完整理论分析，我们识别了与谱偏差和粗粒度表示相关的核心问题，并讨论了如何通过引入相关归纳偏置来缓解这些问题。