Large machine learning models are revolutionary technologies of artificial intelligence whose bottlenecks include huge computational expenses, power, and time used both in the pre-training and fine-tuning process. In this work, we show that fault-tolerant quantum computing could possibly provide provably efficient resolutions for generic (stochastic) gradient descent algorithms, scaling as $\mathcal{O}(T^2 \times \text{polylog}(n))$, where $n$ is the size of the models and $T$ is the number of iterations in the training, as long as the models are both sufficiently dissipative and sparse, with small learning rates. Based on earlier efficient quantum algorithms for dissipative differential equations, we find and prove that similar algorithms work for (stochastic) gradient descent, the primary algorithm for machine learning. In practice, we benchmark instances of large machine learning models from 7 million to 103 million parameters. We find that, in the context of sparse training, a quantum enhancement is possible at the early stage of learning after model pruning, motivating a sparse parameter download and re-upload scheme. Our work shows solidly that fault-tolerant quantum algorithms could potentially contribute to most state-of-the-art, large-scale machine-learning problems.

翻译：大规模机器学习模型是人工智能的革命性技术，其瓶颈包括预训练和微调过程中巨大的计算开销、能耗和时间消耗。本研究表明，容错量子计算可能为通用（随机）梯度下降算法提供可证明的高效解决方案，其复杂度为 $\mathcal{O}(T^2 \times \text{polylog}(n))$，其中 $n$ 为模型规模，$T$ 为训练迭代次数，前提是模型具有充分耗散性和稀疏性，且学习率较小。基于早期针对耗散型微分方程的高效量子算法，我们发现并证明了类似算法适用于机器学习的主要算法——（随机）梯度下降。在实验中，我们对参数规模从700万到1.03亿的大规模机器学习模型实例进行了基准测试。结果表明，在稀疏训练场景下，模型剪枝后的学习初期可实现量子增强，这启发了一种稀疏参数下载与重上传方案。我们的工作坚实证明了容错量子算法有望为当前最先进的大规模机器学习问题做出贡献。