Deep neural networks (DNNs) are becoming progressively large and costly to train. This paper aims to reduce DNN training costs by leveraging preemptible instances on modern clouds, which can be allocated at a much lower price when idle but may be preempted by the cloud provider at any time. Prior work that supports DNN training on preemptive instances employs a reactive approach to handling instance preemptions and allocations after their occurrence, which only achieves limited performance and scalability. We present Parcae, a system that enables cheap, fast, and scalable DNN training on preemptible instances by proactively adjusting the parallelization strategy of a DNN training job to adapt to predicted resource changes before instance preemptions and allocations really happen, which significantly reduces the cost of handling these events. Parcae optimizes liveput, a novel metric that measures the expected training throughput of a DNN job under various possible preemption scenarios. Compared to existing reactive, throughput-optimized systems, Parcae's proactive, live-optimized solution considers both the throughput of a job and its robustness under preemptions. To optimize liveput, Parcae supports lightweight instance migration and uses an availability predictor to forecast future preemptions. It then uses a liveput optimizer to discover an optimal strategy to parallelize DNN training under predicted preemptions. We evaluate Parcae on a variety of DNNs and preemption traces and show that Parcae outperforms existing spot-instance DNN training systems by up to 10$\times$. More importantly, Parcae achieves near-optimal performance for training large DNNs under frequent preemptions, in which case existing approaches cannot make any progress.

翻译：深度神经网络（DNN）的规模日益增大且训练成本高昂。本文旨在通过利用现代云平台上的可抢占实例来降低DNN训练成本——这类实例在空闲时以极低价格分配，但随时可能被云提供商回收。现有支持在可抢占实例上进行DNN训练的工作采用被动式方法处理实例抢占与分配，仅在事件发生后做出反应，导致性能与可扩展性受限。我们提出Parcae系统，通过主动调整DNN训练任务的并行化策略，在实例抢占与分配实际发生前适应预测的资源变化，从而显著降低处理这些事件的成本。Parcae优化了“实时吞吐量”（liveput）这一新型指标，该指标衡量DNN作业在多种可能抢占场景下的预期训练吞吐量。与现有被动式、吞吐量优化的系统相比，Parcae的主动式、实时吞吐量优化方案同时考虑了作业的吞吐量及其在抢占情况下的鲁棒性。为优化实时吞吐量，Parcae支持轻量级实例迁移，并利用可用性预测器预测未来抢占事件。随后，其通过实时吞吐量优化器在预测抢占条件下发现最优的DNN训练并行化策略。我们在多种DNN及抢占轨迹上的评估表明，Parcae的性能较现有spot实例DNN训练系统提升高达10倍。更重要的是，在频繁抢占场景下训练大型DNN时，Parcae实现了近最优性能，而现有方法在此类场景中完全无法取得进展。