Mapping computations to processors and assigning data to memory are critical for maximizing performance in parallel programming. These mapping decisions are managed through the development of specialized low-level system code, called mappers, crafted by performance engineers. Each mapper is tailored to a specific application and optimized for the underlying machine architecture, a process that requires days of refinement and tuning from an expert. Despite advances in system research, automating mapper generation remains a challenge due to the complexity of making millions of decisions to find the optimal solution and generate the solution as code. We introduce an approach that leverages recent advances in LLM-based optimizers for mapper design. In under ten minutes, our method automatically discovers mappers that surpass human expert designs in scientific applications by up to 1.34X speedup. For parallel matrix multiplication algorithms, our mapper achieves up to 1.31X of the expert-designed solution. To achieve this, we simplify the complexity of low-level code generation by introducing a domain-specific language (DSL) that abstracts the low-level system programming details and defines a structured search space for LLMs to explore. To maximize the application performance, we use an LLM optimizer to improve an agentic system that generates the mapper code. As a result, this approach significantly reduces the workload for performance engineers while achieving substantial performance gains across diverse applications. Finally, our results demonstrate the effectiveness of LLM-based optimization in system design and suggest its potential for addressing other complex system challenges.

翻译：在并行编程中，将计算映射至处理器以及将数据分配至内存对于最大化程序性能至关重要。这些映射决策通过开发专门的底层系统代码（称为映射器）来管理，这些代码由性能工程师精心编写。每个映射器都针对特定应用定制，并为底层机器架构进行优化，这一过程需要专家花费数天时间进行反复优化与调优。尽管系统研究取得了进展，但由于需要做出数百万次决策以寻找最优解并生成代码，自动生成映射器仍然是一个挑战。我们提出了一种利用基于LLM的优化器在映射器设计中的最新进展的方法。在十分钟内，我们的方法自动发现的映射器在科学应用中超越了人类专家设计的方案，最高可实现1.34倍的加速。对于并行矩阵乘法算法，我们的映射器达到了专家设计方案的1.31倍性能。为实现这一目标，我们通过引入一种领域特定语言（DSL）来简化底层代码生成的复杂性，该语言抽象了底层系统编程细节，并为LLM定义了一个结构化的搜索空间。为了最大化应用性能，我们使用LLM优化器改进一个生成映射器代码的智能体系统。因此，该方法显著减轻了性能工程师的工作负担，同时在多样化应用中实现了显著的性能提升。最后，我们的结果证明了基于LLM的优化在系统设计中的有效性，并表明其在应对其他复杂系统挑战方面具有潜力。