Large Language Models (LLMs) are capable of generating syntactically correct and functionally complete programs, greatly streamlining software development. However, recent studies reveal that these programs typically execute substantially slower than human-optimized counterparts. Existing approaches to bridging this efficiency gap typically involve either iteratively optimizing code after generation or fine-tuning models on corpora of efficient code. Yet, these methods expose the model to efficiency signals only by mimicking complete, optimized solutions, without explicitly encoding the structural code patterns essential for achieving high runtime performance. Addressing this gap presents two core challenges: (1) extracting and representing latent, efficiency-oriented structural patterns embedded within complex syntax and control flows, and (2) effectively learning these patterns without destabilizing the semantic training of LLMs. To tackle these challenges, we propose EffiSkel, an efficiency skeleton-guided framework that explicitly extracts and learns efficiency skeletons-abstract, reusable structural patterns underpinning efficient code-by leveraging three complementary strategies. These skeletons are integrated into a multi-task learning regime that jointly optimizes code generation and skeleton prediction. Experiments across multiple programming languages and benchmarks demonstrate that EffiSkel significantly enhances both functional correctness and efficiency, resulting on Mercury with DeepSeek-Coder (7B) a +11.11% (vs. EffiCoder) and +3.71% (vs. CodeDPO) higher Efficiency Ratio (ER), and a +0.36 (vs. EffiCoder) and +0.22 (vs. CodeDPO) increase in Average Speedup (AS). These results highlight the effectiveness of explicitly modeling efficiency skeletons in improving the runtime performance of code generated by LLMs.
翻译:大型语言模型(LLMs)能够生成语法正确且功能完整的程序,极大简化了软件开发流程。然而,最新研究表明,这些程序通常比人类优化版本运行速度慢得多。现有解决效率差距的方法主要包括:生成代码后迭代优化,或使用高效代码语料库微调模型。但这些方法仅通过模仿完整的优化方案向模型传递效率信号,并未显式编码实现高性能运行所需的结构化代码模式。针对这一空白,存在两大核心挑战:(1)从复杂语法与控制流中提取并表征蕴含的效率导向结构模式;(2)在不破坏LLMs语义训练稳定性的前提下有效学习这些模式。为应对上述挑战,我们提出EffiSkel——一种效率骨架引导框架,通过三种互补策略显式提取与学习效率骨架(即支撑高效代码的抽象化可复用结构模式)。这些骨架被整合到多任务学习框架中,协同优化代码生成与骨架预测任务。跨多编程语言与基准测试的实验表明,EffiSkel显著提升了功能正确性与执行效率:在Mercury基准上,基于DeepSeek-Coder(7B)的模型相比EffiCoder效率比(ER)提升11.11%,相较CodeDPO提升3.71%;平均加速比(AS)分别提高0.36(vs. EffiCoder)和0.22(vs. CodeDPO)。这些结果凸显了显式建模效率骨架在改善LLM生成代码运行时性能中的有效性。