The evolution of Generative AI and the capabilities of the newly released Large Language Models (LLMs) open new opportunities in software engineering. However, they also lead to new challenges in cybersecurity. Recently, researchers have shown the possibilities of using LLMs such as ChatGPT to generate malicious content that can directly be exploited or guide inexperienced hackers to weaponize tools and code. These studies covered scenarios that still require the attacker to be in the middle of the loop. In this study, we leverage openly available plugins and use an LLM as proxy between the attacker and the victim. We deliver a proof-of-concept where ChatGPT is used for the dissemination of malicious software while evading detection, alongside establishing the communication to a command and control (C2) server to receive commands to interact with a victim's system. Finally, we present the general approach as well as essential elements in order to stay undetected and make the attack a success. This proof-of-concept highlights significant cybersecurity issues with openly available plugins and LLMs, which require the development of security guidelines, controls, and mitigation strategies.

翻译：生成式人工智能的发展以及新发布的大语言模型（LLMs）的能力为软件工程领域带来了新的机遇。然而，它们也在网络安全领域引发了新的挑战。近期，研究人员已展示利用ChatGPT等大语言模型生成可直接利用的恶意内容，或指导缺乏经验的黑客将工具和代码武器化的可能性。这些研究涉及的场景仍要求攻击者处于攻击循环的中间环节。本研究中，我们利用公开可用的插件，将大语言模型作为攻击者与受害者之间的代理。我们实现了一个概念验证系统，该系统使用ChatGPT传播恶意软件以规避检测，同时建立与命令与控制（C2）服务器的通信，接收指令以交互受害者的系统。最后，我们提出了保持隐蔽性并确保攻击成功的一般性方法及关键要素。该概念验证凸显了公开可用插件与大语言模型存在的重大网络安全问题，亟需制定安全指南、控制措施与缓解策略。