AI batch jobs such as model training, inference pipelines, and data analytics require substantial GPU resources and often need to finish before a deadline. Spot instances offer 3-10x lower cost than on-demand instances, but their unpredictable availability makes meeting deadlines difficult. Existing systems either rely solely on spot instances and risk deadline violations, or operate in simplified single-region settings. These approaches overlook substantial spatial and temporal heterogeneity in spot availability, lifetimes, and prices. We show that exploiting such heterogeneity to access more spot capacity is the key to reduce the job execution cost. We present SkyNomad, a multi-region scheduling system that maximizes spot usage and minimizes cost while guaranteeing deadlines. SkyNomad uses lightweight probing to estimate availability, predicts spot lifetimes, accounts for migration cost, and unifies regional characteristics and deadline pressure into a monetary cost model that guides scheduling decisions. Our evaluation shows that SkyNomad achieves 1.25-3.96x cost savings in real cloud deployments and performs within 10% cost differences of an optimal policy in simulation, while consistently meeting deadlines.

翻译：AI批处理作业，如模型训练、推理管道和数据分析，需要大量的GPU资源，并且通常需要在截止时间前完成。竞价实例的成本比按需实例低3-10倍，但其不可预测的可用性使得满足截止时间变得困难。现有系统要么仅依赖竞价实例并面临违反截止时间的风险，要么在简化的单区域设置中运行。这些方法忽略了竞价实例在可用性、生命周期和价格方面存在的显著时空异质性。我们证明，利用这种异质性来获取更多的竞价容量是降低作业执行成本的关键。我们提出了SkyNomad，一个多区域调度系统，它在保证截止时间的同时，最大化竞价实例的使用并最小化成本。SkyNomad使用轻量级探测来估计可用性，预测竞价实例的生命周期，考虑迁移成本，并将区域特性与截止时间压力统一到一个指导调度决策的货币成本模型中。我们的评估表明，SkyNomad在实际云部署中实现了1.25-3.96倍的成本节省，在模拟中其成本与最优策略的差异在10%以内，同时始终满足截止时间。