While pre-trained language models (LM) for code have achieved great success in code completion, they generate code conditioned only on the contents within the file, i.e., in-file context, but ignore the rich semantics in other files within the same project, i.e., cross-file context, a critical source of information that is especially useful in modern modular software development. Such overlooking constrains code language models' capacity in code completion, leading to unexpected behaviors such as generating hallucinated class member functions or function calls with unexpected arguments. In this work, we develop a cross-file context finder tool, CCFINDER, that effectively locates and retrieves the most relevant cross-file context. We propose CoCoMIC, a framework that incorporates cross-file context to learn the in-file and cross-file context jointly on top of pretrained code LMs. CoCoMIC successfully improves the existing code LM with a 33.94% relative increase in exact match and a 28.69% relative increase in identifier matching for code completion when the cross-file context is provided.

翻译：尽管预训练语言模型在代码补全任务上取得了巨大成功，但它们生成代码时仅以文件内内容（即文件内上下文）为条件，忽视了同一项目中其他文件中的丰富语义（即跨文件上下文），而这一关键信息源在现代模块化软件开发中尤为有用。这种忽略限制了代码语言模型在代码补全中的能力，导致出现诸如生成虚构的类成员函数或带有意外参数的函数调用等异常行为。在本工作中，我们开发了一个跨文件上下文查找工具CCFINDER，能够有效定位并检索最相关的跨文件上下文。我们提出了CoCoMIC框架，该框架在预训练代码语言模型基础上，通过融合跨文件上下文来联合学习文件内和跨文件上下文。当提供跨文件上下文时，CoCoMIC成功提升了现有代码语言模型的性能，在代码补全的精确匹配指标上相对提升33.94%，在标识符匹配指标上相对提升28.69%。