Spear-phishing attacks present a significant security challenge, with large language models (LLMs) escalating the threat by generating convincing emails and facilitating target reconnaissance. To address this, we propose a detection approach based on a novel document vectorization method that utilizes an ensemble of LLMs to create representation vectors. By prompting LLMs to reason and respond to human-crafted questions, we quantify the presence of common persuasion principles in the email's content, producing prompted contextual document vectors for a downstream supervised machine learning model. We evaluate our method using a unique dataset generated by a proprietary system that automates target reconnaissance and spear-phishing email creation. Our method achieves a 91% F1 score in identifying LLM-generated spear-phishing emails, with the training set comprising only traditional phishing and benign emails. Key contributions include an innovative document vectorization method utilizing LLM reasoning, a publicly available dataset of high-quality spear-phishing emails, and the demonstrated effectiveness of our method in detecting such emails. This methodology can be utilized for various document classification tasks, particularly in adversarial problem domains.

翻译：鱼叉式网络钓鱼攻击构成重大安全挑战，而大型语言模型（LLMs）通过生成逼真邮件并辅助目标侦查进一步加剧了这一威胁。为此，我们提出一种基于新型文档向量化方法的检测方案——利用LLMs集成生成表示向量。通过提示LLMs对人类构建的问题进行推理与作答，我们量化邮件内容中常见说服原则的存在程度，从而为下游监督式机器学习模型生成提示式上下文文档向量。我们采用由专有系统生成的独特数据集进行评估——该系统可自动化目标侦查与鱼叉式网络钓鱼邮件生成。该方法在识别LLM生成的鱼叉式网络钓鱼邮件时达到91%的F1分数，且训练集仅包含传统钓鱼邮件与良性邮件。核心贡献包括：利用LLM推理的创新文档向量化方法、公开可用的高质量鱼叉式网络钓鱼邮件数据集，以及该方法在检测此类邮件中的有效性验证。本方法可适用于各类文档分类任务，尤其在对抗性问题上具有独特优势。