Large language models (LLMs) have demonstrated impressive results on natural language tasks, and security researchers are beginning to employ them in both offensive and defensive systems. In cyber-security, there have been multiple research efforts that utilize LLMs focusing on the pre-breach stage of attacks like phishing and malware generation. However, so far there lacks a comprehensive study regarding whether LLM-based systems can be leveraged to simulate the post-breach stage of attacks that are typically human-operated, or "hands-on-keyboard" attacks, under various attack techniques and environments. As LLMs inevitably advance, they may be able to automate both the pre- and post-breach attack stages. This shift may transform organizational attacks from rare, expert-led events to frequent, automated operations requiring no expertise and executed at automation speed and scale. This risks fundamentally changing global computer security and correspondingly causing substantial economic impacts, and a goal of this work is to better understand these risks now so we can better prepare for these inevitable ever-more-capable LLMs on the horizon. On the immediate impact side, this research serves three purposes. First, an automated LLM-based, post-breach exploitation framework can help analysts quickly test and continually improve their organization's network security posture against previously unseen attacks. Second, an LLM-based penetration test system can extend the effectiveness of red teams with a limited number of human analysts. Finally, this research can help defensive systems and teams learn to detect novel attack behaviors preemptively before their use in the wild....

翻译：大语言模型在自然语言任务中展现出令人瞩目的成果，安全研究人员已开始将其应用于攻防系统。在网络安全领域，已有诸多研究利用大语言模型聚焦于攻击的前入侵阶段，例如钓鱼攻击和恶意软件生成。然而，目前尚缺乏全面研究探讨：基于大语言模型的系统是否能够模拟通常需要人工操作的后入侵阶段攻击（即"键盘操作式"攻击），并适配不同攻击技术与环境。随着大语言模型不可避免的进步，它们或能实现前入侵与后入侵攻击阶段的全面自动化。这种转变可能将针对组织的攻击行为，从罕见、由专家主导的事件，转变为频繁、无需专业知识、以自动化速度和规模执行的常规操作。这或将从根本上改变全球计算机安全格局，并引发显著的经济影响。本研究的核心目标正是为了更深入理解这些风险，以便为即将来临的、功能日益强大的大语言模型做好应对准备。就当前影响层面而言，本研究旨在实现三个目标：第一，基于大语言模型的自动化后入侵利用框架，可帮助分析人员快速测试并持续改善组织网络针对未知攻击的安全态势；第二，基于大语言模型的渗透测试系统，能扩展人力分析人员有限的蓝队团队的效能；第三，本研究可帮助防御系统与团队提前学习检测新型攻击行为，防患于未然。