With the rapid evolution of large language models (LLMs), the demand for automated, high-performance system kernels has emerged as a key enabler for accelerating development and deployment. We introduce TritonRL, a domain-specialized LLM for Triton kernel generation, trained with a novel training framework that enables robust and automated kernel synthesis. Unlike general-purpose programming languages, Triton kernel generation faces unique challenges due to data scarcity and incomplete evaluation criteria, vulnerable to reward hacking. Our approach addresses these challenges end-to-end by distilling Triton-specific knowledge through supervised fine-tuning on curated datasets, and further improving code quality via reinforcement learning (RL) with robust, verifiable rewards and hierarchical reward assignment. Our RL framework robustly detects reward hacking and guides both reasoning traces and code tokens through fine-grained verification and hierarchical reward decomposition, enabling the model to generate high-quality Triton kernels that can truly replace existing modules. With robust and fine-grained evaluation, our experiments on KernelBench demonstrate that TritonRL achieves state-of-the-art correctness and speedup, surpassing all other Triton-specific models and underscoring the effectiveness of our RL-based training paradigm.

翻译：随着大型语言模型（LLMs）的快速发展，对自动化高性能系统内核的需求已成为加速开发与部署的关键推动力。我们提出了TritonRL，一个专门用于Triton内核生成的领域专用LLM，它采用一种新颖的训练框架进行训练，能够实现鲁棒且自动化的内核合成。与通用编程语言不同，Triton内核生成因数据稀缺和评估标准不完整而面临独特挑战，容易受到奖励欺骗的影响。我们的方法通过端到端的方式应对这些挑战：首先在精心策划的数据集上进行监督微调以提炼Triton特定知识，然后通过强化学习（RL）结合鲁棒、可验证的奖励及分层奖励分配机制来进一步提升代码质量。我们的RL框架能够鲁棒地检测奖励欺骗，并通过细粒度验证和分层奖励分解来引导推理轨迹和代码标记，使模型能够生成真正可替代现有模块的高质量Triton内核。通过鲁棒且细粒度的评估，我们在KernelBench上的实验表明，TritonRL在正确性和加速比方面达到了最先进的水平，超越了所有其他Triton专用模型，并突显了我们基于RL的训练范式的有效性。