Fault Localization (FL) aims to automatically localize buggy lines of code, a key first step in many manual and automatic debugging tasks. Previous FL techniques assume the provision of input tests, and often require extensive program analysis, program instrumentation, or data preprocessing. Prior work on deep learning for APR struggles to learn from small datasets and produces limited results on real-world programs. Inspired by the ability of large language models (LLMs) of code to adapt to new tasks based on very few examples, we investigate the applicability of LLMs to line level fault localization. Specifically, we propose to overcome the left-to-right nature of LLMs by fine-tuning a small set of bidirectional adapter layers on top of the representations learned by LLMs to produce LLMAO, the first language model based fault localization approach that locates buggy lines of code without any test coverage information. We fine-tune LLMs with 350 million, 6 billion, and 16 billion parameters on small, manually curated corpora of buggy programs such as the Defects4J corpus. We observe that our technique achieves substantially more confidence in fault localization when built on the larger models, with bug localization performance scaling consistently with the LLM size. Our empirical evaluation shows that LLMAO improves the Top-1 results over the state-of-the-art machine learning fault localization (MLFL) baselines by 2.3%-54.4%, and Top-5 results by 14.4%-35.6%. LLMAO is also the first FL technique trained using a language model architecture that can detect security vulnerabilities down to the code line level.

翻译：故障定位旨在自动定位存在缺陷的代码行，这是许多手动和自动调试任务中的关键第一步。以往的故障定位技术假设提供输入测试用例，通常需要大量程序分析、程序插桩或数据预处理。先前基于深度学习的自动程序修复研究难以从小规模数据集中学习，且对真实世界程序效果有限。受代码大语言模型能基于极少量示例适应新任务的能力启发，我们探究了大语言模型在线级故障定位中的适用性。具体而言，我们提出通过在大语言模型所学习的表示基础上微调一组小型双向适配器层，以克服大语言模型从左到右的局限性，进而提出首个无需任何测试覆盖信息即可定位缺陷代码行的基于语言模型的故障定位方法——LLMAO。我们在人工整理的小规模缺陷程序语料库（如Defects4J）上，对参数量为3.5亿、60亿和160亿的大语言模型进行微调。实验发现，基于更大模型构建的故障定位方法在置信度上显著提升，且缺陷定位性能随大语言模型规模扩大而持续提升。实证评估表明，LLMAO在最先进机器学习故障定位基线方法的基础上，Top-1准确率提升2.3%~54.4%，Top-5准确率提升14.4%~35.6%。此外，LLMAO是首个采用语言模型架构训练、能精确定位至代码行级安全漏洞的故障定位技术。