Large Language Models are expanding beyond being a tool humans use and into independent agents that can observe an environment, reason about solutions to problems, make changes that impact those environments, and understand how their actions impacted their environment. One of the most common applications of these LLM Agents is in computer programming, where agents can successfully work alongside humans to generate code while controlling programming environments or networking systems. However, with the increasing ability and complexity of these agents comes dangers about the potential for their misuse. A concerning application of LLM agents is in the domain cybersecurity, where they have the potential to greatly expand the threat imposed by attacks such as social engineering. This is due to the fact that LLM Agents can work autonomously and perform many tasks that would normally require time and effort from skilled human programmers. While this threat is concerning, little attention has been given to assessments of the capabilities of LLM coding agents in generating code for social engineering attacks. In this work we compare different LLMs in their ability and willingness to produce potentially dangerous code bases that could be misused by cyberattackers. The result is a dataset of 200 website code bases and logs from 40 different LLM coding agents. Analysis of models shows which metrics of LLMs are more and less correlated with performance in generating spear-phishing sites. Our analysis and the dataset we present will be of interest to researchers and practitioners concerned in defending against the potential misuse of LLMs in spear-phishing.

翻译：大型语言模型正从人类使用的工具演变为能够独立观察环境、推理问题解决方案、实施影响环境的变更，并能理解自身行为如何影响环境的智能代理。这类LLM代理最常见的应用领域之一是计算机编程，它们能够成功与人类协作生成代码，同时控制编程环境或网络系统。然而，随着这些代理能力与复杂度的提升，其被滥用的潜在风险也日益凸显。LLM代理在网络安全领域的应用尤其令人担忧，它们可能极大扩展社会工程学等攻击手段的威胁范围。这是因为LLM代理能够自主工作，执行许多通常需要熟练人类程序员投入大量时间和精力的任务。尽管这种威胁令人不安，但针对LLM编程代理生成社会工程学攻击代码能力的评估研究仍鲜有关注。本研究通过比较不同LLM生成可能被网络攻击者滥用的危险代码库的能力与倾向，构建了包含200个网站代码库及40种不同LLM编程代理运行日志的数据集。模型分析揭示了LLM的哪些指标与生成鱼叉式钓鱼网站的表现存在更高或更低的相关性。我们的分析及所提供的数据集将为关注防御LLM在鱼叉式网络钓鱼中潜在滥用的研究人员和实践者提供重要参考。