Large language models (LLMs) for code have become indispensable in various domains, including code generation, reasoning tasks and agent systems. While open-access code LLMs are increasingly approaching the performance levels of proprietary models, high-quality code LLMs suitable for rigorous scientific investigation, particularly those with reproducible data processing pipelines and transparent training protocols, remain limited. The scarcity is due to various challenges, including resource constraints, ethical considerations, and the competitive advantages of keeping models advanced. To address the gap, we introduce OpenCoder, a top-tier code LLM that not only achieves performance comparable to leading models but also serves as an "open cookbook" for the research community. Unlike most prior efforts, we release not only model weights and inference code, but also the reproducible training data, complete data processing pipeline, rigorous experimental ablation results, and detailed training protocols for open scientific research. Through this comprehensive release, we identify the key ingredients for building a top-tier code LLM: (1) code optimized heuristic rules for data cleaning and methods for data deduplication, (2) recall of text corpus related to code and (3) high-quality synthetic data in both annealing and supervised fine-tuning stages. By offering this level of openness, we aim to broaden access to all aspects of a top-tier code LLM, with OpenCoder serving as both a powerful model and an open foundation to accelerate research, and enable reproducible advancements in code AI.

翻译：面向代码的大语言模型（LLMs）已在代码生成、推理任务和智能体系统等多个领域变得不可或缺。尽管开源代码大语言模型正逐步接近专有模型的性能水平，但适用于严格科学研究的优质代码大语言模型——尤其是那些具备可复现数据处理流程和透明训练方案的模型——仍然有限。这种稀缺性源于多重挑战，包括资源限制、伦理考量以及保持模型先进性的竞争优势。为填补这一空白，我们推出了OpenCoder，一个不仅性能可比肩领先模型，更能作为研究社区“开放食谱”的顶级代码大语言模型。与大多数先前工作不同，我们不仅公开了模型权重和推理代码，还发布了可复现的训练数据、完整的数据处理流程、严谨的实验消融结果以及详细的训练方案，以支持开放科学研究。通过这一全面发布，我们明确了构建顶级代码大语言模型的关键要素：（1）针对代码优化的数据清洗启发式规则与数据去重方法，（2）与代码相关文本语料的召回，以及（3）在退火阶段与监督微调阶段均使用高质量合成数据。通过提供这种程度的开放性，我们旨在拓宽对顶级代码大语言模型所有层面的访问，使OpenCoder既作为强大模型，也作为开放基础平台，以加速代码人工智能领域的研究并推动可复现的进步。