Resource-management tasks in modern operating and distributed systems continue to rely primarily on hand-designed heuristics for tasks such as scheduling, caching, or active queue management. Designing performant heuristics is an expensive, time-consuming process that we are forced to continuously go through due to the constant flux of hardware, workloads and environments. We propose a new alternative: synthesizing instance-optimal heuristics -- specialized for the exact workloads and hardware where they will be deployed -- using code-generating large language models (LLMs). To make this synthesis tractable, Vulcan separates policy and mechanism through LLM-friendly, task-agnostic interfaces. With these interfaces, users specify the inputs and objectives of their desired policy, while Vulcan searches for performant policies via evolutionary search over LLM-generated code. This interface is expressive enough to capture a wide range of system policies, yet sufficiently constrained to allow even small, inexpensive LLMs to generate correct and executable code. We use Vulcan to synthesize performant heuristics for cache eviction and memory tiering, and find that these heuristics outperform all human-designed state-of-the-art algorithms by upto 69% and 7.9% in performance for each of these tasks respectively.

翻译：现代操作系统与分布式系统中的资源管理任务（如调度、缓存或主动队列管理）仍主要依赖于人工设计的启发式方法。设计高性能启发式方法是一个昂贵且耗时的过程，而由于硬件、工作负载与环境持续变化，我们不得不反复进行这一过程。我们提出一种新的替代方案：利用代码生成型大语言模型（LLMs）合成实例最优的启发式方法——专为部署时的具体工作负载和硬件定制。为使该合成过程可行，Vulcan通过LLM友好、任务无关的接口实现策略与机制的分离。借助这些接口，用户可指定目标策略的输入与目标，而Vulcan则通过基于LLM生成代码的进化搜索来寻找高性能策略。该接口具有足够表达能力以涵盖各类系统策略，同时又受到充分约束，使得即使小型、低成本的LLMs也能生成正确且可执行的代码。我们使用Vulcan合成了缓存淘汰与内存分层的高性能启发式方法，发现这些启发式方法在两项任务中分别以最高69%和7.9%的性能优势超越了所有人设计的最先进算法。