As one of the most popular dynamic languages, Python experiences a decrease in readability and maintainability when code smells are present. Recent advancements in Large Language Models have sparked growing interest in AI-enabled tools for both code generation and refactoring. GitHub Copilot is one such tool that has gained widespread usage. Copilot Chat, released on September 2023, functions as an interactive tool aims at facilitating natural language-powered coding. However, limited attention has been given to understanding code smells in Copilot-generated Python code and Copilot's ability to fix the code smells it generates. To this end, we built a dataset comprising 102 code smells in Copilot-generated Python code. Our aim is to first explore the occurrence of code smells in Copilot-generated Python code and then evaluate the effectiveness of Copilot in fixing these code smells employing different prompts. The results show that 8 out of 10 types of Python smells can be detected in Copilot-generated Python code, among which Multiply-Nested Container is the most common one. For these code smells, Copilot Chat achieves a highest fixing rate of 87.1%, showing promise in fixing Python code smells generated by Copilot itself. Besides, the effectiveness of Copilot Chat in fixing these smells can be improved with the provision of more detailed prompts. However, using Copilot Chat to fix these smells might introduce new code smells.
翻译:作为最流行的动态语言之一,Python 在存在代码异味时其可读性和可维护性会有所下降。大型语言模型的最新进展引发了人们对基于人工智能的代码生成与重构工具日益增长的兴趣。GitHub Copilot 就是其中一种已广泛使用的工具。2023 年 9 月发布的 Copilot Chat 旨在作为一种交互式工具,促进基于自然语言的编程。然而,目前对于 Copilot 生成的 Python 代码中代码异味的研究,以及 Copilot 修复其自身生成代码异味的能力关注有限。为此,我们构建了一个包含 102 个 Copilot 生成 Python 代码中代码异味的数据集。我们的目标是首先探究 Copilot 生成的 Python 代码中代码异味的出现情况,然后评估 Copilot 使用不同提示修复这些代码异味的有效性。结果表明,在 Copilot 生成的 Python 代码中可检测到 10 种 Python 代码异味中的 8 种,其中多重嵌套容器最为常见。针对这些代码异味,Copilot Chat 的最高修复率达到 87.1%,显示出其在修复 Copilot 自身生成的 Python 代码异味方面的潜力。此外,提供更详细的提示可以提高 Copilot Chat 修复这些异味的有效性。然而,使用 Copilot Chat 修复这些异味可能会引入新的代码异味。