The exponential growth in demand for GPU computing resources has created an urgent need for automated CUDA optimization strategies. While recent advances in LLMs show promise for code generation, current SOTA models achieve low success rates in improving CUDA speed. In this paper, we introduce CUDA-L1, an automated reinforcement learning framework for CUDA optimization that employs a novel contrastive RL algorithm. CUDA-L1 achieves significant performance improvements on the CUDA optimization task: trained on A100, it delivers an average speedup of x3.12 with a median speedup of x1.42 against default baselines over across all 250 CUDA kernels of KernelBench, with peak speedups reaching x120. In addition to the default baseline provided by KernelBench, CUDA-L1 demonstrates x2.77 over Torch Compile, x2.88 over Torch Compile with reduce overhead, x2.81 over CUDA Graph implementations, and x7.72 over cuDNN libraries. Furthermore, the model also demonstrates portability across different GPU architectures. Beyond these benchmark results, CUDA-L1 demonstrates several properties: it 1) discovers a variety of CUDA optimization techniques and learns to combine them strategically to achieve optimal performance; 2) uncovers fundamental principles of CUDA optimization, such as the multiplicative nature of optimizations; 3) identifies non-obvious performance bottlenecks and rejects seemingly beneficial optimizations that actually harm performance. The capabilities demonstrate that, RL can transform an initially poor-performing LLM into an effective CUDA optimizer through speedup-based reward signals alone, without human expertise or domain knowledge. This paradigm opens possibilities for automated optimization of CUDA operations, and holds promise to substantially promote GPU efficiency and alleviate the rising pressure on GPU computing resources. Project: deepreinforce-ai.github.io/cudal1_blog

翻译：GPU计算资源需求的指数级增长催生了对自动化CUDA优化策略的迫切需求。尽管近期大语言模型在代码生成方面展现出潜力，但当前最先进模型在提升CUDA运行速度方面成功率较低。本文提出CUDA-L1——一种采用新型对比强化学习算法的自动化CUDA优化强化学习框架。CUDA-L1在CUDA优化任务上实现了显著的性能提升：在A100显卡上训练后，其在KernelBench全部250个CUDA内核测试中，相较于默认基线实现了平均3.12倍、中位数1.42倍的加速效果，峰值加速比达到120倍。除KernelBench提供的默认基线外，CUDA-L1相较于Torch Compile实现2.77倍加速，相较于降低开销的Torch Compile实现2.88倍加速，相较于CUDA Graph实现2.81倍加速，相较于cuDNN库实现7.72倍加速。此外，该模型还展现出跨GPU架构的移植能力。除基准测试结果外，CUDA-L1还展现出以下特性：1）能够发现多种CUDA优化技术并学会策略性地组合它们以实现最优性能；2）揭示了CUDA优化的基本原理，如优化效果的乘性累积特性；3）能识别非显性性能瓶颈，并拒绝那些看似有益实则损害性能的优化方案。这些能力表明，仅通过基于加速比的奖励信号，强化学习即可将初始性能欠佳的大语言模型转化为有效的CUDA优化器，无需人类专家经验或领域知识。该范式为CUDA运算的自动化优化开辟了新途径，有望显著提升GPU效率并缓解日益增长的GPU计算资源压力。项目地址：deepreinforce-ai.github.io/cudal1_blog