Code completion (CC) is a task frequently used by developers when working in collaboration with LLM-based programming assistants. Despite the increased performance of LLMs on public benchmarks, out of the box LLMs still have a hard time generating code that aligns with a private code repository not previously seen by the model's training data. Customizing code LLMs to a private repository provides a way to improve the model performance. In this paper we present our approach for automated LLM customization based on semantic scopes in the code. We evaluate LLMs on real industry cases with two private enterprise code repositories with two customization strategies: Retrieval-Augmented Generation (RAG) and supervised Fine-Tuning (FT). Our mechanism for ingesting the repository's data and formulating the training data pairs with semantic scopes helps models to learn the underlying patterns specific to the repository, providing more precise code to developers and helping to boost their productivity. The code completions of moderately sized customized models can be significantly better than those of uncustomized models of much larger capacity. We also include an analysis of customization on two public benchmarks and present opportunities for future work.

翻译：代码补全（CC）是开发人员在使用基于大语言模型的编程助手时频繁执行的任务。尽管大语言模型在公开基准测试上的性能有所提升，但未经定制的现成大语言模型在生成与模型训练数据中未曾见过的私有代码库相一致的代码时，仍然面临困难。将代码大语言模型定制到私有代码库为提高模型性能提供了一种途径。本文提出了一种基于代码中语义作用域的自动化大语言模型定制方法。我们在两个私有企业代码库的真实工业案例中，使用两种定制策略评估了大语言模型：检索增强生成（RAG）和监督式微调（FT）。我们提出的代码库数据摄取机制以及利用语义作用域构建训练数据对的方法，有助于模型学习特定于该代码库的底层模式，从而为开发人员提供更精确的代码并助力提升其生产力。经过定制的适度规模模型所产生的代码补全结果，可以显著优于未经定制的、规模大得多的模型。我们还包含了对两个公开基准测试的定制效果分析，并展望了未来的研究方向。