Large machine learning models are revolutionary technologies of artificial intelligence whose bottlenecks include huge computational expenses, power, and time used both in the pre-training and fine-tuning process. In this work, we show that fault-tolerant quantum computing could possibly provide provably efficient resolutions for generic (stochastic) gradient descent algorithms, scaling as O(T^2 polylog(n)), where n is the size of the models and T is the number of iterations in the training, as long as the models are both sufficiently dissipative and sparse, with small learning rates. Based on earlier efficient quantum algorithms for dissipative differential equations, we find and prove that similar algorithms work for (stochastic) gradient descent, the primary algorithm for machine learning. In practice, we benchmark instances of large machine learning models from 7 million to 103 million parameters. We find that, in the context of sparse training, a quantum enhancement is possible at the early stage of learning after model pruning, motivating a sparse parameter download and re-upload scheme. Our work shows solidly that fault-tolerant quantum algorithms could potentially contribute to most state-of-the-art, large-scale machine-learning problems.

翻译：大型机器学习模型是人工智能的革命性技术，其瓶颈包括预训练和微调过程中巨大的计算开销、功耗和时间消耗。本研究证明，只要模型具有足够的耗散性和稀疏性且学习率较小，容错量子计算可能为通用（随机）梯度下降算法提供可证明的高效解决方案，其复杂度为O(T² polylog(n))，其中n为模型规模，T为训练迭代次数。基于早期针对耗散微分方程的高效量子算法，我们发现并证明了类似算法适用于机器学习的主要算法——（随机）梯度下降。在实证中，我们对参数规模从700万到1.03亿的大型机器学习模型实例进行基准测试，发现在稀疏训练场景下，模型剪枝后的学习初期可实现量子加速，由此提出稀疏参数下载与重上传方案。本研究坚实证明了容错量子算法有望为当前最先进的大规模机器学习问题做出贡献。