Artificial intelligence has achieved significant success in handling complex tasks in recent years. This success is due to advances in machine learning algorithms and hardware acceleration. In order to obtain more accurate results and solve more complex problems, algorithms must be trained with more data. This huge amount of data could be time-consuming to process and require a great deal of computation. This solution could be achieved by distributing the data and algorithm across several machines, which is known as distributed machine learning. There has been considerable effort put into distributed machine learning algorithms, and different methods have been proposed so far. In this article, we present a comprehensive summary of the current state-of-the-art in the field through the review of these algorithms. We divide this algorithms in classification and clustering (traditional machine learning), deep learning and deep reinforcement learning groups. Distributed deep learning has gained more attention in recent years and most of studies worked on this algorithms. As a result, most of the articles we discussed here belong to this category. Based on our investigation of algorithms, we highlight limitations that should be addressed in future research.

翻译：人工智能近年来在处理复杂任务方面取得了显著成功。这一成功得益于机器学习算法的进步和硬件加速。为了获得更精确的结果并解决更复杂的问题，算法必须使用更多数据进行训练。海量数据的处理可能耗时且需要大量计算。解决方案之一是将数据和算法分布到多台机器上，即分布式机器学习。目前已有大量关于分布式机器学习算法的研究，且提出了多种不同的方法。本文通过综述这些算法，全面总结了该领域的当前最新进展。我们将这些算法分为分类与聚类（传统机器学习）、深度学习以及深度强化学习三大类。近年来，分布式深度学习受到更多关注，大部分研究聚焦于此。因此，本文讨论的多数文章属于这一类别。基于对算法的分析，我们指出了未来研究亟需解决的局限性。