Stochastic optimisation algorithms are the de facto standard for machine learning with large amounts of data. Handling only a subset of available data in each optimisation step dramatically reduces the per-iteration computational costs, while still ensuring significant progress towards the solution. Driven by the need to solve large-scale optimisation problems as efficiently as possible, the last decade has witnessed an explosion of research in this area. Leveraging the parallels between machine learning and inverse problems has allowed harnessing the power of this research wave for solving inverse problems. In this survey, we provide a comprehensive account of the state-of-the-art in stochastic optimisation from the viewpoint of inverse problems. We present algorithms with diverse modalities of problem randomisation and discuss the roles of variance reduction, acceleration, higher-order methods, and other algorithmic modifications, and compare theoretical results with practical behaviour. We focus on the potential and the challenges for stochastic optimisation that are unique to inverse imaging problems and are not commonly encountered in machine learning. We conclude the survey with illustrative examples from imaging problems to examine the advantages and disadvantages that this new generation of algorithms bring to the field of inverse problems.

翻译：随机优化算法已成为处理海量数据机器学习的事实标准。通过在每次优化迭代中仅处理可用数据的子集，该方法显著降低了单次迭代的计算成本，同时仍能确保向解方向取得显著进展。在尽可能高效求解大规模优化问题的需求驱动下，过去十年该领域研究呈现爆炸式增长。借助机器学习与逆问题之间的相似性，这股研究浪潮的力量得以被应用于逆问题求解。本综述从逆问题的视角，对随机优化领域的最新进展进行全面阐述。我们展示了具有不同问题随机化模式的算法，探讨了方差缩减、加速技术、高阶方法及其他算法改进的作用，并将理论结果与实际性能进行对比。我们特别关注随机优化在逆成像问题中特有的潜力与挑战——这些是机器学习领域不常遇到的。最后，我们通过成像问题的示例案例，审视新一代算法为逆问题领域带来的优势与局限。