We release Code Llama, a family of large language models for code based on Llama 2 providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks. We provide multiple flavors to cover a wide range of applications: foundation models (Code Llama), Python specializations (Code Llama - Python), and instruction-following models (Code Llama - Instruct) with 7B, 13B, 34B and 70B parameters each. All models are trained on sequences of 16k tokens and show improvements on inputs with up to 100k tokens. 7B, 13B and 70B Code Llama and Code Llama - Instruct variants support infilling based on surrounding content. Code Llama reaches state-of-the-art performance among open models on several code benchmarks, with scores of up to 67% and 65% on HumanEval and MBPP, respectively. Notably, Code Llama - Python 7B outperforms Llama 2 70B on HumanEval and MBPP, and all our models outperform every other publicly available model on MultiPL-E. We release Code Llama under a permissive license that allows for both research and commercial use.

翻译：我们发布了Code Llama，这是一系列基于Llama 2的代码大型语言模型，在开放模型中实现了最先进的性能，具备代码填充能力、对大输入上下文的支持，以及编程任务的零样本指令跟随能力。我们提供多种变体以覆盖广泛的应用场景：基础模型（Code Llama）、Python专用模型（Code Llama - Python）和指令跟随模型（Code Llama - Instruct），分别拥有7B、13B、34B和70B的参数规模。所有模型均在16k token的序列上进行训练，并对长达100k token的输入表现出改进效果。7B、13B和70B版本的Code Llama及Code Llama - Instruct变体支持基于周围内容的代码填充。Code Llama在多个代码基准测试中达到了开放模型中的最先进性能，在HumanEval和MBPP上分别取得了高达67%和65%的得分。值得注意的是，Code Llama - Python 7B在HumanEval和MBPP上超越了Llama 2 70B，并且我们所有模型在MultiPL-E上均优于其他任何公开可用的模型。我们以允许研究及商业用途的宽松许可协议发布Code Llama。