Over the past few years, Large Language Models of Code (Code LLMs) have started to have a significant impact on programming practice. Code LLMs are also emerging as a building block for research in programming languages and software engineering. However, the quality of code produced by a Code LLM varies significantly by programming languages. Code LLMs produce impressive results on programming languages that are well represented in their training data (e.g., Java, Python, or JavaScript), but struggle with low-resource languages, like OCaml and Racket. This paper presents an effective approach for boosting the performance of Code LLMs on low-resource languages using semi-synthetic data. Our approach generates high-quality datasets for low-resource languages, which can then be used to fine-tune any pretrained Code LLM. Our approach, called MultiPL-T, translates training data from high-resource languages into training data for low-resource languages. We apply our approach to generate tens of thousands of new, validated training items for Racket, OCaml, and Lua from Python. Moreover, we use an open dataset (The Stack) and model (StarCoderBase), which allow us to decontaminate benchmarks and train models on this data without violating the model license. With MultiPL-T generated data, we present fine-tuned versions of StarCoderBase that achieve state-of-the-art performance for Racket, OCaml, and Lua on benchmark problems. For Lua, our fine-tuned model achieves the same performance as StarCoderBase as Python -- a very high-resource language -- on the MultiPL-E benchmarks. For Racket and OCaml, we double their performance on MultiPL-E, bringing their performance close to higher-resource languages such as Ruby and C#.

翻译：过去几年中，代码大语言模型（Code LLMs）已开始对编程实践产生重大影响。Code LLMs 正逐步成为编程语言与软件工程研究的基础组件。然而，Code LLM 生成的代码质量因编程语言不同而差异显著。对于训练数据中充分覆盖的编程语言（如 Java、Python 或 JavaScript），Code LLMs 能产生令人印象深刻的结果，但在处理低资源语言（如 OCaml 和 Racket）时则表现欠佳。本文提出了一种有效方法，利用半合成数据提升 Code LLMs 在低资源语言上的性能。该方法可为低资源语言生成高质量数据集，进而用于微调任意预训练的 Code LLM。我们提出的方法名为 MultiPL-T，它将高资源语言的训练数据翻译为低资源语言的训练数据。我们应用该方法从 Python 数据中为 Racket、OCaml 和 Lua 生成了数万个经过验证的新训练项。此外，我们使用开放数据集（The Stack）和模型（StarCoderBase），这使得我们能够对基准测试进行数据去污染，并在不违反模型许可协议的前提下利用这些数据训练模型。借助 MultiPL-T 生成的数据，我们推出了 StarCoderBase 的微调版本，其在 Racket、OCaml 和 Lua 的基准测试问题上达到了最先进性能。对于 Lua，我们的微调模型在 MultiPL-E 基准测试上达到了与 StarCoderBase 在 Python（一种极高资源语言）上相同的性能。对于 Racket 和 OCaml，我们在 MultiPL-E 上将其性能提升了一倍，使它们的表现接近 Ruby 和 C# 等高资源语言。