Compiler error messages serve as an initial resource for programmers dealing with compilation errors. However, previous studies indicate that they often lack sufficient targeted information to resolve code issues. Consequently, programmers typically rely on their own research to fix errors. Historically, Stack Overflow has been the primary resource for such information, but recent advances in large language models offer alternatives. This study systematically examines 100 compiler error messages from three sources to determine the most effective approach for programmers encountering compiler errors. Factors considered include Stack Overflow search methods and the impact of model version and prompt phrasing when using large language models. The results reveal that GPT-4 outperforms Stack Overflow in explaining compiler error messages, the effectiveness of adding code snippets to Stack Overflow searches depends on the search method, and results for Stack Overflow differ significantly between Google and StackExchange API searches. Furthermore, GPT-4 surpasses GPT-3.5, with "How to fix" prompts yielding superior outcomes to "What does this error mean" prompts. These results offer valuable guidance for programmers seeking assistance with compiler error messages, underscoring the transformative potential of advanced large language models like GPT-4 in debugging and opening new avenues of exploration for researchers in AI-assisted programming.
翻译:编译器错误信息是程序员处理编译错误时首先利用的资源。然而,以往的研究表明,这些信息往往缺乏足够的针对性,不足以解决代码问题。因此,程序员通常依赖自身调研来修正错误。历史上,Stack Overflow一直是此类信息的主要来源,但近期大语言模型的进展提供了替代方案。本研究系统考察了来自三个来源的100条编译器错误信息,以确定程序员在遇到编译器错误时最有效的方式。考虑的因素包括Stack Overflow的搜索方法,以及使用大语言模型时模型版本和提示措辞的影响。结果显示,GPT-4在解释编译器错误信息方面优于Stack Overflow;在Stack Overflow搜索中加入代码片段的有效性取决于搜索方法;且Stack Overflow在Google搜索与StackExchange API搜索之间的结果存在显著差异。此外,GPT-4优于GPT-3.5,且“如何修复”提示相比“此错误含义”提示能产生更优结果。这些结果为寻求编译器错误信息帮助的程序员提供了有价值的指导,凸显了GPT-4等先进大语言模型在调试中的变革潜力,并为AI辅助编程领域的研究者开辟了新的探索方向。