Large Language Models (LLMs) have recently attracted wide interest for tackling software engineering tasks. In contrast to code generation, refactoring demands precise, semantics-preserving edits that improve program structure, which also makes automated evaluation challenging. However, existing refactoring benchmarks commonly suffer from three shortcomings: limited coverage of refactoring scenarios, the inclusion of instances that mix refactoring with unrelated changes, and insufficient repository-level context for realistic assessment. To mitigate these issues, we introduce SWE-Refactor, a new benchmark for LLM-based code refactoring. SWE-Refactor comprises 1,099 developer-written, behavior-preserving refactorings mined from 18 Java projects, including 922 atomic and 177 compound instances. Each instance is validated via compilation, test execution, and automated refactoring detection tools to ensure correctness. We evaluate nine widely used LLMs on SWE-Refactor, covering models such as GPT-4o-mini, DeepSeek-V3, and CodeLLaMa, to provide representative reference results. Our results show that complex and compound refactorings remain the primary source of failures; notably, an OpenAI Codex agent achieves only 39.4% success on compound instances. We release SWE-Refactor and all evaluation results to facilitate future research on LLM-based code refactoring.

翻译：大型语言模型（LLM）最近在解决软件工程任务方面引起了广泛关注。与代码生成相比，重构要求进行精确、保持语义的编辑以改进程序结构，这也使得自动化评估具有挑战性。然而，现有的重构基准测试普遍存在三个不足：重构场景覆盖有限、包含将重构与无关更改混合的实例，以及缺乏用于现实评估的仓库级上下文。为了缓解这些问题，我们引入了SWE-Refactor，这是一个用于基于LLM的代码重构的新基准测试。SWE-Refactor包含从18个Java项目中挖掘出的1,099个由开发者编写的、保持行为的重构实例，其中包括922个原子重构和177个复合重构实例。每个实例都通过编译、测试执行和自动化重构检测工具进行验证，以确保其正确性。我们在SWE-Refactor上评估了九个广泛使用的LLM，涵盖诸如GPT-4o-mini、DeepSeek-V3和CodeLLaMa等模型，以提供具有代表性的参考结果。我们的结果表明，复杂和复合重构仍然是失败的主要来源；值得注意的是，一个OpenAI Codex智能体在复合实例上仅实现了39.4%的成功率。我们发布SWE-Refactor及所有评估结果，以促进未来基于LLM的代码重构研究。