Recent advancements in code-fluent Large Language Models (LLMs) enabled the research on repository-level code editing. In such tasks, the model navigates and modifies the entire codebase of a project according to request. Hence, such tasks require efficient context retrieval, i.e., navigating vast codebases to gather relevant context. Despite the recognized importance of context retrieval, existing studies tend to approach repository-level coding tasks in an end-to-end manner, rendering the impact of individual components within these complicated systems unclear. In this work, we decouple the task of context retrieval from the other components of the repository-level code editing pipelines. We lay the groundwork to define the strengths and weaknesses of this component and the role that reasoning plays in it by conducting experiments that focus solely on context retrieval. We conclude that while the reasoning helps to improve the precision of the gathered context, it still lacks the ability to identify its sufficiency. We also outline the ultimate role of the specialized tools in the process of context gathering. The code supplementing this paper is available at https://github.com/JetBrains-Research/ai-agents-code-editing.
翻译:近年来,代码流畅的大型语言模型(LLMs)的进展推动了代码库级代码编辑的研究。在此类任务中,模型需根据请求导航并修改项目的整个代码库。因此,这类任务需要高效的上下文检索,即通过导航庞大的代码库以收集相关上下文。尽管上下文检索的重要性已得到公认,现有研究往往以端到端的方式处理代码库级编码任务,导致这些复杂系统中各组成部分的影响尚不明确。在本工作中,我们将上下文检索任务从代码库级代码编辑流程的其他组件中解耦出来。通过专注于上下文检索的实验,我们为界定该组件的优势与不足以及推理在其中扮演的角色奠定了基础。我们的结论是:虽然推理有助于提高所收集上下文的精确度,但其仍缺乏判断上下文充分性的能力。我们还概述了专用工具在上下文收集过程中的根本作用。本文的补充代码可在 https://github.com/JetBrains-Research/ai-agents-code-editing 获取。