In the task of code translation, neural network-based models have been shown to frequently produce semantically erroneous code that deviates from the original logic of the source code. This issue persists even with advanced large models. Although a recent approach proposed using test cases to identify these semantic errors, it relies heavily on the quality of the test cases and is not applicable to code snippets without test cases in real-world scenarios. Therefore, We present EISP, a static analysis framework based on the Large Language Model (LLM).First, the framework generates a semantic mapping between source code and translated code. Next, each sub-code fragment is identified by recursively traversing the abstract syntax tree of the source code, and its corresponding translated code fragment is found through the semantic mapping. Finally, EISP connects each pair of sub-code fragments with fine-grained knowledge hints through an AI chain to assist LLMs in discovering semantic mistakes in the translated code. In our benchmark evaluation, the EISP framework, based on GPT-4o mini, achieved an accuracy of 82.3\%, representing a 20.3\% improvement over baseline methods using the same base model, and a 7.4\% improvement compared to dynamic analysis methods that require test cases and manual intervention. To our knowledge, EISP is the first tool to locate semantic errors in translated code without test cases or compilable code. This innovative tool provides the software engineering community with a new way to deal with code fragments without test cases.
翻译:在代码翻译任务中,基于神经网络的模型经常会产生语义错误的代码,这些代码偏离了源代码的原始逻辑。即使使用先进的大模型,这一问题依然存在。尽管最近有方法提出使用测试用例来识别这些语义错误,但该方法严重依赖测试用例的质量,且不适用于现实场景中缺乏测试用例的代码片段。因此,我们提出了EISP,一个基于大语言模型的静态分析框架。首先,该框架生成源代码与翻译代码之间的语义映射。接着,通过递归遍历源代码的抽象语法树识别每个子代码片段,并通过语义映射找到其对应的翻译代码片段。最后,EISP通过AI链将每对子代码片段与细粒度知识提示相连接,以协助大语言模型发现翻译代码中的语义错误。在我们的基准评估中,基于GPT-4o mini的EISP框架达到了82.3%的准确率,相比使用相同基础模型的基线方法提升了20.3%,与需要测试用例和人工干预的动态分析方法相比提升了7.4%。据我们所知,EISP是首个无需测试用例或可编译代码即可定位翻译代码中语义错误的工具。这一创新工具为软件工程社区提供了一种处理无测试用例代码片段的新方法。