The rapid advancement of workflows and methods for software engineering using AI emphasizes the need for a systematic evaluation and analysis of their ability to leverage information from entire projects, particularly in large code bases. In this challenge on optimization of context collection for code completion, organized by JetBrains in collaboration with Mistral AI as part of the ASE 2025 conference, participants developed efficient mechanisms for collecting context from source code repositories to improve fill-in-the-middle code completions for Python and Kotlin. We constructed a large dataset of real-world code in these two programming languages using permissively licensed open-source projects. The submissions were evaluated based on their ability to maximize completion quality for multiple state-of-the-art neural models using the chrF metric. During the public phase of the competition, nineteen teams submitted solutions to the Python track and eight teams submitted solutions to the Kotlin track. In the private phase, six teams competed, of which five submitted papers to the workshop.

翻译：随着利用人工智能进行软件工程的工作流程和方法快速发展，亟需对其利用整个项目信息（尤其是在大型代码库中）的能力进行系统性评估与分析。本次由JetBrains与Mistral AI合作组织、作为ASE 2025会议组成部分的代码补全上下文收集优化挑战中，参赛者开发了从源代码仓库高效收集上下文的机制，以提升Python和Kotlin语言的中间填充式代码补全效果。我们基于采用宽松许可证的开源项目，构建了包含这两种编程语言真实代码的大规模数据集。参赛方案根据其使用chrF指标优化多种前沿神经模型补全质量的能力进行评估。在竞赛公开阶段，十九支团队提交了Python赛道方案，八支团队提交了Kotlin赛道方案。在私有阶段，六支团队参与角逐，其中五支向研讨会提交了论文。