The evolution of Generative AI and the capabilities of the newly released Large Language Models (LLMs) open new opportunities in software engineering. However, they also lead to new challenges in cybersecurity. Recently, researchers have shown the possibilities of using LLMs such as ChatGPT to generate malicious content that can directly be exploited or guide inexperienced hackers to weaponize tools and code. Those studies covered scenarios that still require the attacker in the middle of the loop. In this study, we leverage openly available plugins and use an LLM as proxy between the attacker and the victim. We deliver a proof-of-concept where ChatGPT is used for the dissemination of malicious software while evading detection, alongside establishing the communication to a command and control (C2) server to receive commands to interact with a victim's system. Finally, we present the general approach as well as essential elements in order to stay undetected and make the attack a success. This proof-of-concept highlights significant cybersecurity issues with openly available plugins and LLMs, which require the development of security guidelines, controls, and mitigation strategies.

翻译：生成式人工智能的演进以及新发布的大语言模型（LLMs）的能力，为软件工程领域带来了新的机遇。然而，它们也引发了网络安全方面的新挑战。近期，研究人员已经展示了利用ChatGPT等大语言模型生成可直接被利用的恶意内容，或指导经验不足的黑客将工具和代码武器化的可能性。这些研究涵盖了仍需攻击者介入循环的场景。在本研究中，我们利用公开可用的插件，将大语言模型作为攻击者与受害者之间的代理。我们提供了一个概念验证，展示如何使用ChatGPT在逃避检测的同时传播恶意软件，并建立与命令与控制（C2）服务器的通信以接收指令，从而与受害者系统进行交互。最后，我们提出了保持隐蔽并确保攻击成功的一般方法及关键要素。这一概念验证凸显了公开可用插件与大语言模型所导致的重大网络安全问题，亟需制定安全指南、控制措施及缓解策略。