Recent advances in Large Language Models (LLMs) have shown significant potential in enhancing cybersecurity defenses against sophisticated threats. LLM-based penetration testing is an essential step in automating system security evaluations by identifying vulnerabilities. Remediation, the subsequent crucial step, addresses these discovered vulnerabilities. Since details about vulnerabilities, exploitation methods, and software versions offer crucial insights into system weaknesses, integrating penetration testing with vulnerability remediation into a cohesive system has become both intuitive and necessary. This paper introduces PenHeal, a two-stage LLM-based framework designed to autonomously identify and mitigate security vulnerabilities. The framework integrates two LLM-enabled components: the Pentest Module, which detects multiple vulnerabilities within a system, and the Remediation Module, which recommends optimal remediation strategies. The integration is facilitated through Counterfactual Prompting and an Instructor module that guides the LLMs using external knowledge to explore multiple potential attack paths effectively. Our experimental results demonstrate that PenHeal not only automates the identification and remediation of vulnerabilities but also significantly improves vulnerability coverage by 31%, increases the effectiveness of remediation strategies by 32%, and reduces the associated costs by 46% compared to baseline models. These outcomes highlight the transformative potential of LLMs in reshaping cybersecurity practices, offering an innovative solution to defend against cyber threats.

翻译：近年来，大语言模型（LLMs）的进展在增强针对复杂威胁的网络安全防御方面展现出巨大潜力。基于LLM的渗透测试通过识别漏洞，成为自动化系统安全评估的关键步骤。修复作为后续的关键步骤，旨在解决这些已发现的漏洞。由于关于漏洞、利用方法及软件版本的详细信息为系统弱点提供了关键洞察，将渗透测试与漏洞修复整合为一个连贯的系统既直观又必要。本文提出了PenHeal，一个设计用于自主识别并缓解安全漏洞的两阶段LLM框架。该框架集成了两个由LLM驱动的组件：渗透测试模块（用于检测系统中的多个漏洞）和修复模块（用于推荐最优修复策略）。两者通过反事实提示和一个指导模块实现集成，该指导模块利用外部知识引导LLM，以有效探索多条潜在攻击路径。我们的实验结果表明，与基线模型相比，PenHeal不仅实现了漏洞识别与修复的自动化，还将漏洞覆盖率显著提高了31%，修复策略的有效性提升了32%，同时将相关成本降低了46%。这些成果凸显了LLM在重塑网络安全实践方面的变革潜力，为防御网络威胁提供了一种创新解决方案。