We introduce control models for LLM-powered code completion in JetBrains IDEs: ML classifiers which trigger inference and filter the generated suggestions to better align them with users and reduce unnecessary requests. To this end, we evaluate boosting- and transformer-based architectures on an offline dataset of real code completions with n=98 users. We further evaluate the offline classification performance of our boosting-based approach on a range of syntactically diverse languages; and perform an A/B study in a production environment where they improve completion efficiency and quality metrics. With this study, we hope to demonstrate the potential in using auxiliary models for smarter in-IDE integration of LLM-driven features, highlight fruitful future directions, and open problems.

翻译：本文介绍了JetBrains集成开发环境（IDE）中基于大语言模型（LLM）的代码补全控制模型：这是一种机器学习分类器，用于触发推理并过滤生成的建议，以更好地适应用户需求并减少不必要的请求。为此，我们在一个包含n=98位用户的真实代码补全离线数据集上评估了基于Boosting和Transformer的架构。我们进一步评估了基于Boosting的方法在多种语法多样化的语言上的离线分类性能；并在生产环境中进行了A/B测试，结果表明该方法提升了补全效率与质量指标。通过此项研究，我们希望展示辅助模型在实现更智能的IDE内LLM驱动功能集成方面的潜力，指明富有前景的未来研究方向，并揭示有待解决的开放性问题。