In this work we consider stochastic gradient descent (SGD) for solving linear inverse problems in Banach spaces. SGD and its variants have been established as one of the most successful optimisation methods in machine learning, imaging and signal processing, etc. At each iteration SGD uses a single datum, or a small subset of data, resulting in highly scalable methods that are very attractive for large-scale inverse problems. Nonetheless, the theoretical analysis of SGD-based approaches for inverse problems has thus far been largely limited to Euclidean and Hilbert spaces. In this work we present a novel convergence analysis of SGD for linear inverse problems in general Banach spaces: we show the almost sure convergence of the iterates to the minimum norm solution and establish the regularising property for suitable a priori stopping criteria. Numerical results are also presented to illustrate features of the approach.

翻译：本文考虑随机梯度下降法求解巴拿赫空间中的线性反问题。随机梯度下降及其变体已被确立为机器学习、成像与信号处理等领域最成功的优化方法之一。每次迭代中，随机梯度下降仅使用单个数据点或小子集数据，这使得该方法具有高度可扩展性，对大规模反问题极具吸引力。然而，目前基于随机梯度下降的反问题理论分析大多局限于欧几里得空间和希尔伯特空间。本文提出了一般巴拿赫空间中随机梯度下降法求解线性反问题的新收敛性分析：我们证明了迭代序列几乎必然收敛到最小范数解，并针对合适的先验停止准则建立了正则化性质。文中还给出了数值结果以说明该方法的特点。