Cluster resource allocation is a multidimensional search problem that finds the best allocation of tasks to servers. Because the search space grows exponentially, modern approaches frame it as a mixed integer program (MIP) or a complex set of search heuristics. This paper proposes using a different approach: convex optimization, which has extremely fast solution methods. The research challenge is devising how to transform cluster resource allocation into a convex problem that generates good placements. We describe CvxCluster, which allocates cluster resources with a two-stage algorithm. The first stage solves a convex relaxation of the placement problem to yield a principled set of per-machine resource prices. The second stage uses these prices to drive a lightweight greedy procedure to place tasks. Experimental results with Azure traces find that CvxCluster scales to 100,480 servers under proportional workload growth and sustains arrival rates up to 500,000x the baseline trace. CvxCluster runs 100 to 2,500x faster than a state-of-the-art MIP solver while remaining within 3% of the optimal objective. CvxCluster can support complex constraints such as job anti-affinity, machine types, and GPU servers. The key insight behind CvxCluster is that reformulating placement as a continuous rather than discrete problem enables much faster methods that find solutions just as good or better than prior heuristics.

翻译：摘要：集群资源分配是一个多维搜索问题，旨在寻找任务到服务器的最优分配方案。由于搜索空间呈指数级增长，现代方法通常将其建模为混合整数规划（MIP）或复杂的搜索启发式算法。本文提出采用不同的方法：凸优化，该方法具有极快的求解速度。研究挑战在于如何将集群资源分配转化为能生成优质放置方案的凸问题。我们描述了CvxCluster，它通过两阶段算法分配集群资源。第一阶段求解放置问题的凸松弛，生成一组有原则的每台机器资源价格。第二阶段利用这些价格驱动轻量级贪心过程来放置任务。使用Azure轨迹的实验结果表明，在工作负载按比例增长的情况下，CvxCluster可扩展至100,480台服务器，并能够维持高达基线轨迹500,000倍的到达速率。CvxCluster的运行速度比最先进的MIP求解器快100至2,500倍，同时目标函数值保持在最优解的3%以内。CvxCluster支持作业反亲和性、机器类型和GPU服务器等复杂约束。CvxCluster的核心见解在于：将放置问题从离散形式重构为连续形式，能够实现更快速的求解方法，其解质量与以往启发式算法相当甚至更优。