While the pay-as-you-go nature of cloud virtual machines (VMs) makes it easy to spin-up large clusters for training ML models, it can also lead to ballooning costs. The 100s of virtual machine sizes provided by cloud platforms also makes it extremely challenging to select the ``right'' cloud cluster configuration for training. Furthermore, the training time and cost of distributed model training is highly sensitive to the cluster configurations, and presents a large and complex tradeoff-space. In this paper, we develop principled and practical techniques for optimizing the training time and cost of distributed ML model training on the cloud. Our key insight is that both parallel and statistical efficiency must be considered when selecting the optimum job configuration parameters such as the number of workers and the batch size. By combining conventional parallel scaling concepts and new insights into SGD noise, our models accurately estimate the time and cost on different cluster configurations with < 5% error. Using the repetitive nature of training and our models, we can search for optimum cloud configurations in a black-box, online manner. Our approach reduces training times by 2 times and costs more more than 50%. Compared to an oracle-based approach, our performance models are accurate to within 2% such that the search imposes an overhead of just 10%.

翻译：摘要：尽管云虚拟机（VMs）的按需付费特性使得搭建大规模集群进行机器学习模型训练变得便捷，但也可能导致成本激增。云平台提供的数百种虚拟机规格使得为训练选择“正确”的云集群配置极具挑战性。此外，分布式模型训练的训练时间与成本对集群配置高度敏感，呈现出庞大而复杂的权衡空间。本文提出了优化云端分布式机器学习模型训练时间与成本的原理性与实用性技术。我们的关键洞察在于：选择最优作业配置参数（如工作节点数量与批量大小）时，必须同时考虑并行效率与统计效率。通过结合传统并行扩展理论和对随机梯度下降（SGD）噪声的新见解，我们的模型能以低于5%的误差精准估算不同集群配置下的训练时间与成本。利用训练过程的重复性及所提模型，我们能够以黑盒在线方式搜索最优云配置。该方法可将训练时间缩短至原来的1/2，成本降低超过50%。与基于基准方法的方案相比，我们的性能模型精度误差在2%以内，且搜索产生的额外开销仅为10%。