We release Code Llama, a family of large language models for code based on Llama 2 providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks. We provide multiple flavors to cover a wide range of applications: foundation models (Code Llama), Python specializations (Code Llama - Python), and instruction-following models (Code Llama - Instruct) with 7B, 13B and 34B parameters each. All models are trained on sequences of 16k tokens and show improvements on inputs with up to 100k tokens. 7B and 13B Code Llama and Code Llama - Instruct variants support infilling based on surrounding content. Code Llama reaches state-of-the-art performance among open models on several code benchmarks, with scores of up to 53% and 55% on HumanEval and MBPP, respectively. Notably, Code Llama - Python 7B outperforms Llama 2 70B on HumanEval and MBPP, and all our models outperform every other publicly available model on MultiPL-E. We release Code Llama under a permissive license that allows for both research and commercial use.

翻译：我们发布了Code Llama，这是一个基于Llama 2的代码大语言模型系列，在开源模型中提供了最先进的性能，具备代码补全能力、对大输入上下文的支持以及编程任务中的零样本指令跟随能力。我们提供了多种模型变体以覆盖广泛的应用场景：基础模型（Code Llama）、Python专用模型（Code Llama - Python）以及指令跟随模型（Code Llama - Instruct），参数规模分别为7B、13B和34B。所有模型均使用16k token序列进行训练，并对高达100k token的输入展现性能提升。7B和13B版本的Code Llama与Code Llama - Instruct变体支持基于上下文内容的代码补全。Code Llama在多个代码基准测试中达到了开源模型的最优性能，在HumanEval和MBPP上分别取得了高达53%和55%的得分。值得注意的是，Code Llama - Python 7B在HumanEval和MBPP上超越了Llama 2 70B，且我们所有模型在MultiPL-E上的表现均优于其他所有公开可用模型。我们以允许研究和商业用途的宽松许可协议发布了Code Llama。