We release Code Llama, a family of large language models for code based on Llama 2 providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks. We provide multiple flavors to cover a wide range of applications: foundation models (Code Llama), Python specializations (Code Llama - Python), and instruction-following models (Code Llama - Instruct) with 7B, 13B and 34B parameters each. All models are trained on sequences of 16k tokens and show improvements on inputs with up to 100k tokens. 7B and 13B Code Llama and Code Llama - Instruct variants support infilling based on surrounding content. Code Llama reaches state-of-the-art performance among open models on several code benchmarks, with scores of up to 53% and 55% on HumanEval and MBPP, respectively. Notably, Code Llama - Python 7B outperforms Llama 2 70B on HumanEval and MBPP, and all our models outperform every other publicly available model on MultiPL-E. We release Code Llama under a permissive license that allows for both research and commercial use.

翻译：我们发布了Code Llama（代码羊驼），这是一个基于Llama 2构建的代码大型语言模型系列，在开源模型中提供了最先进的性能，具备代码补全能力、支持大输入上下文，以及编程任务的零样本指令跟随能力。我们提供了多种版本以覆盖广泛的应用场景：基础模型（Code Llama）、Python专用模型（Code Llama - Python）以及指令跟随模型（Code Llama - Instruct），每个版本分别包含7B、13B和34B参数。所有模型均在16k词元序列上进行训练，并在多达100k词元的输入上展现出改进效果。7B和13B的Code Llama及Code Llama - Instruct版本支持基于上下文内容的代码补全。Code Llama在多个代码基准测试中达到了开源模型中的最先进性能，在HumanEval和MBPP上分别取得了高达53%和55%的得分。值得注意的是，Code Llama - Python 7B在HumanEval和MBPP上超越了Llama 2 70B，并且我们所有模型在MultiPL-E上的表现均优于其他所有公开可用模型。我们以宽松许可协议发布Code Llama，允许研究及商业用途。