Legacy programming languages such as COBOL (Common Business-Oriented Language) remain critical in business computing. However, maintaining legacy COBOL systems is increasingly challenging due to a declining pool of skilled developers and the persistence of COBOL errors that require deep domain expertise to resolve. This paper investigates the challenges of COBOL compilation errors and introduces a framework leveraging large language models (LLMs) to address these issues. We first categorize the common compilation errors in LLM-generated COBOL code into three groups: incomplete code errors, syntax errors, and type-related errors. We further propose COBOLAssist, a technique to enhance code correctness through iterative repairs guided by compilation feedback. Our evaluation using five LLMs including GPT variants and mAInframer, shows a high prevalence of incorrect program structures and function usage in COBOL programs and demonstrates the effectiveness of COBOLAssist, with the compilation success rates increasing from 29.5\% to 64.38\% for GPT-4o-mini and from 41.8\% to 95.89\% for GPT-4o. It also improves pass@1 significantly, for example from 9.1 to 22.6 for GPT-4. Notably, while mAInframer-34B achieves the highest compilation success rate, its functional correctness remains limited. This research not only highlights the limitations in current LLMs for COBOL but also demonstrates a practical path forward for automated debugging in legacy systems.

翻译：以COBOL（通用商业导向语言）为代表的传统编程语言在商业计算领域仍至关重要。然而，随着具备深厚领域知识的熟练开发人员日益稀缺，遗留COBOL系统中的错误修复愈发困难。本文探究COBOL编译错误的挑战，并提出一个利用大语言模型（LLM）解决此类问题的框架。我们首先将LLM生成的COBOL代码中的常见编译错误归纳为三类：代码不完整错误、语法错误和类型相关错误。进一步提出COBOLAssist技术，通过基于编译反馈的迭代修复提升代码正确性。基于包括GPT系列和mAInframer在内的五个LLM的评估表明，COBOL程序中存在大量错误程序结构和函数使用问题，同时验证了COBOLAssist的有效性：GPT-4o-mini的编译成功率从29.5%提升至64.38%，GPT-4o从41.8%提升至95.89%。该方法还显著提高了pass@1指标，例如GPT-4的pass@1从9.1提升至22.6。值得注意的是，尽管mAInframer-34B取得了最高的编译成功率，但其功能正确性仍存在局限。本研究不仅揭示了当前LLM在COBOL领域的局限性，也为遗留系统的自动化调试提供了切实可行的路径。