Phishing, whether through email, SMS, or malicious websites, poses a major threat to organizations by using social engineering to trick users into revealing sensitive information. It not only compromises company's data security but also incurs significant financial losses. In this paper, we investigate whether the remarkable performance of Large Language Models (LLMs) can be leveraged for particular task like text classification, particularly detecting malicious content and compare its results with state-of-the-art Deberta V3 (DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing) model. We systematically assess the potential and limitations of both approaches using comprehensive public datasets comprising diverse data sources such as email, HTML, URL, SMS, and synthetic data generation. Additionally, we demonstrate how LLMs can generate convincing phishing emails, making it harder to spot scams and evaluate the performance of both models in this context. Our study delves further into the challenges encountered by DeBERTa V3 during its training phases, fine-tuning methodology and transfer learning processes. Similarly, we examine the challenges associated with LLMs and assess their respective performance. Among our experimental approaches, the transformer-based DeBERTa method emerged as the most effective, achieving a test dataset (HuggingFace phishing dataset) recall (sensitivity) of 95.17% closely followed by GPT-4 providing a recall of 91.04%. We performed additional experiments with other datasets on the trained DeBERTa V3 model and LLMs like GPT 4 and Gemini 1.5. Based on our findings, we provide valuable insights into the effectiveness and robustness of these advanced language models, offering a detailed comparative analysis that can inform future research efforts in strengthening cybersecurity measures for detecting and mitigating phishing threats.

翻译：钓鱼攻击，无论是通过电子邮件、短信还是恶意网站，都利用社会工程学诱骗用户泄露敏感信息，对组织机构构成重大威胁。它不仅危害企业的数据安全，还会造成重大的经济损失。本文研究了能否利用大语言模型（LLMs）在文本分类等特定任务（尤其是恶意内容检测）中的卓越性能，并将其结果与最先进的DeBERTa V3（采用ELECTRA风格预训练及梯度解耦嵌入共享的DeBERTa）模型进行比较。我们使用包含电子邮件、HTML、URL、短信及合成数据生成等多种数据源的综合性公共数据集，系统评估了两种方法的潜力与局限性。此外，我们展示了LLMs如何生成具有高度迷惑性的钓鱼邮件，从而增加诈骗识别难度，并在此背景下评估了两种模型的性能。本研究进一步深入探讨了DeBERTa V3在训练阶段、微调方法及迁移学习过程中遇到的挑战。同样地，我们分析了LLMs面临的挑战并评估了其相应性能。在我们的实验方法中，基于Transformer的DeBERTa方法表现最为有效，在测试数据集（HuggingFace钓鱼数据集）上实现了95.17%的召回率（敏感度），紧随其后的是GPT-4，召回率为91.04%。我们在训练好的DeBERTa V3模型以及GPT-4和Gemini 1.5等LLMs上使用其他数据集进行了补充实验。基于研究结果，我们对这些先进语言模型的有效性和鲁棒性提供了有价值的见解，并提供了详细的对比分析，可为未来加强网络安全措施以检测和缓解钓鱼威胁的研究工作提供参考。