Large language models (LLMs) have shown remarkable progress in code generation, but their generated code often suffers from inefficiency, resulting in longer execution times and higher memory consumption. To address this issue, we propose Self Optimization based on OverheAd Profile (SOAP), a self-optimization framework that utilizes execution overhead profiles to improve the efficiency of LLM-generated code. SOAP first generates code using an LLM, then executes it locally to capture execution time and memory usage profiles. These profiles are fed back to the LLM, which then revises the code to reduce overhead. To evaluate the effectiveness of SOAP, we conduct extensive experiments on the EffiBench, HumanEval, and MBPP with 16 open-source and 6 closed-source models. Our evaluation results demonstrate that through iterative self-optimization, SOAP significantly enhances the efficiency of LLM-generated code. For example, the execution time (ET) of StarCoder2-15B for the EffiBench decreases from 0.93 (s) to 0.12 (s) which reduces 87.1% execution time requirement compared with the initial code. The total memory usage (TMU) of StarCoder2-15B also decreases from 22.02 (Mb*s) to 2.03 (Mb*s), which decreases 90.8% total memory consumption during the execution process. The source code of SOAP was released in https://github.com/huangd1999/SOAP.

翻译：大型语言模型（LLM）在代码生成方面取得了显著进展，但其生成的代码往往存在效率低下的问题，导致执行时间更长、内存消耗更高。为解决这一问题，我们提出了基于开销剖析的自优化框架SOAP，该框架利用执行开销剖析来提升LLM生成代码的效率。SOAP首先使用LLM生成代码，随后在本地执行以捕获执行时间和内存使用剖析数据。这些剖析数据被反馈给LLM，进而驱动模型对代码进行修订以降低开销。为评估SOAP的有效性，我们在EffiBench、HumanEval和MBPP基准上对16个开源模型和6个闭源模型进行了广泛实验。评估结果表明，通过迭代自优化，SOAP能显著提升LLM生成代码的效率。例如，在EffiBench上，StarCoder2-15B生成代码的执行时间从0.93秒降至0.12秒，相比初始代码减少了87.1%的执行时间需求；其总内存使用量也从22.02（Mb*秒）降至2.03（Mb*秒），在执行过程中减少了90.8%的总内存消耗。SOAP的源代码已发布于https://github.com/huangd1999/SOAP。