The ability of ChatGPT to generate human-like responses and understand context has made it a popular tool for conversational agents, content creation, data analysis, and research and innovation. However, its effectiveness and ease of accessibility makes it a prime target for generating malicious content, such as phishing attacks, that can put users at risk. In this work, we identify several malicious prompts that can be provided to ChatGPT to generate functional phishing websites. Through an iterative approach, we find that these phishing websites can be made to imitate popular brands and emulate several evasive tactics that have been known to avoid detection by anti-phishing entities. These attacks can be generated using vanilla ChatGPT without the need of any prior adversarial exploits (jailbreaking).

翻译：ChatGPT生成类人回复和理解上下文的能力使其成为对话代理、内容创作、数据分析以及研究创新领域的热门工具。然而，其高效性和易获取性也使其成为生成恶意内容的重点目标，例如可能使用户面临风险的网络钓鱼攻击。在本研究中，我们识别出若干可提供给ChatGPT的恶意提示，用以生成功能性钓鱼网站。通过迭代方法，我们发现这些钓鱼网站能够模仿知名品牌，并采用多种已知可逃避反钓鱼实体检测的规避策略。这些攻击无需任何先前的对抗性利用（越狱）即可通过原生ChatGPT生成。