The rise of Large Language Models (LLMs) has significantly advanced many applications on software engineering tasks, particularly in code generation. Despite the promising performance, LLMs are prone to generate hallucinations, which means LLMs might produce outputs that deviate from users' intent, exhibit internal inconsistencies, or misalign with the factual knowledge, making the deployment of LLMs potentially risky in a wide range of applications. Existing work mainly focuses on investing the hallucination in the domain of natural language generation (NLG), leaving a gap in understanding the types and extent of hallucinations in the context of code generation. To bridge the gap, we conducted a thematic analysis of the LLM-generated code to summarize and categorize the hallucinations present in it. Our study established a comprehensive taxonomy of hallucinations in LLM-generated code, encompassing 5 primary categories of hallucinations depending on the conflicting objectives and varying degrees of deviation observed in code generation. Furthermore, we systematically analyzed the distribution of hallucinations, exploring variations among different LLMs and their correlation with code correctness. Based on the results, we proposed HalluCode, a benchmark for evaluating the performance of code LLMs in recognizing hallucinations. Hallucination recognition and mitigation experiments with HalluCode and HumanEval show existing LLMs face great challenges in recognizing hallucinations, particularly in identifying their types, and are hardly able to mitigate hallucinations. We believe our findings will shed light on future research about hallucination evaluation, detection, and mitigation, ultimately paving the way for building more effective and reliable code LLMs in the future.

翻译：大语言模型（LLMs）的兴起显著推进了软件工程任务中诸多应用的发展，尤其在代码生成领域表现突出。尽管性能令人瞩目，但LLMs容易产生幻觉，即其输出可能偏离用户意图、出现内部不一致或与事实知识不符，这使得LLMs在广泛应用中存在潜在风险。现有研究主要聚焦于自然语言生成（NLG）领域的幻觉问题，而对代码生成场景下幻觉的类型与程度缺乏理解。为填补这一空白，我们对LLM生成的代码进行了主题分析，总结并分类了其中存在的幻觉。我们的研究建立了LLM生成代码中幻觉的全面分类体系，涵盖五大主要类别，这些类别基于代码生成过程中观察到的冲突目标和不同程度的偏离。此外，我们系统分析了幻觉的分布，探索了不同LLM之间的差异及其与代码正确性的关联。基于研究结果，我们提出了HalluCode，一个用于评估代码LLM幻觉识别能力的基准。通过HalluCode和HumanEval进行的幻觉识别与缓解实验表明，现有LLM在识别幻觉（尤其是判断其类型）方面面临巨大挑战，且几乎无法缓解幻觉。我们相信，这一发现将为未来关于幻觉评估、检测和缓解的研究提供启示，最终为构建更有效、更可靠的代码LLM铺平道路。