As coding challenges become more complex, recent advancements in Large Language Models (LLMs) have led to notable successes, such as achieving a 94.6\% solve rate on the HumanEval benchmark. Concurrently, there is an increasing commercial push for repository-level inline code completion tools, such as GitHub Copilot and Tab Nine, aimed at enhancing developer productivity. This paper delves into the transition from individual coding problems to repository-scale solutions, presenting a thorough review of the current literature on effective LLM prompting for code generation at the repository level. We examine approaches that will work with black-box LLMs such that they will be useful and applicable to commercial use cases, and their applicability in interpreting code at a repository scale. We juxtapose the Repository-Level Prompt Generation technique with RepoCoder, an iterative retrieval and generation method, to highlight the trade-offs inherent in each approach and to establish best practices for their application in cutting-edge coding benchmarks. The interplay between iterative refinement of prompts and the development of advanced retrieval systems forms the core of our discussion, offering a pathway to significantly improve LLM performance in code generation tasks. Insights from this study not only guide the application of these methods but also chart a course for future research to integrate such techniques into broader software engineering contexts.
翻译:随着编程挑战日益复杂,大型语言模型的最新进展已取得显著成果,例如在HumanEval基准测试中达到94.6%的解题率。与此同时,以GitHub Copilot和Tab Nine为代表的仓库级内联代码补全工具在商业领域加速推广,旨在提升开发者生产力。本文深入探讨从独立编程问题向仓库规模解决方案的范式转变,系统梳理当前关于大语言模型在仓库级代码生成中有效提示方法的文献。我们聚焦于能够与黑盒大语言模型兼容的方法,使其适用于商业场景,并评估其在仓库规模代码理解中的可应用性。通过将仓库级提示生成技术与迭代检索生成方法RepoCoder进行对比分析,揭示了两种方法在核心权衡方面的差异,并确立了其在先进编程基准测试中的最佳实践。围绕提示的迭代优化与先进检索系统开发的相互作用构成了本文讨论的核心,为显著提升大语言模型在代码生成任务中的性能提供了可行路径。本研究不仅为这些方法的应用提供了指导,更勾勒了未来在更广泛软件工程环境中集成此类技术的研究方向。