Phishing attacks attempt to deceive users into stealing sensitive information, posing a significant cybersecurity threat. Advances in machine learning (ML) and deep learning (DL) have led to the development of numerous phishing webpage detection solutions, but these models remain vulnerable to adversarial attacks. Evaluating their robustness against adversarial phishing webpages is essential. Existing tools contain datasets of pre-designed phishing webpages for a limited number of brands, and lack diversity in phishing features. To address these challenges, we develop PhishOracle, a tool that generates adversarial phishing webpages by embedding diverse phishing features into legitimate webpages. We evaluate the robustness of two existing models, Stack model and Phishpedia, in classifying PhishOracle-generated adversarial phishing webpages. Additionally, we study a commercial large language model, Gemini Pro Vision, in the context of adversarial attacks. We conduct a user study to determine whether PhishOracle-generated adversarial phishing webpages deceive users. Our findings reveal that many PhishOracle-generated phishing webpages evade current phishing webpage detection models and deceive users, but Gemini Pro Vision is robust to the attack. We also develop the PhishOracle web app, allowing users to input a legitimate URL, select relevant phishing features and generate a corresponding phishing webpage. All resources are publicly available on GitHub.

翻译：钓鱼攻击试图欺骗用户以窃取敏感信息，构成重大的网络安全威胁。机器学习（ML）和深度学习（DL）的进步催生了众多钓鱼网页检测解决方案，但这些模型仍易受对抗攻击的影响。评估它们对抗对抗性钓鱼网页的鲁棒性至关重要。现有工具包含针对有限数量品牌预先设计的钓鱼网页数据集，且钓鱼特征缺乏多样性。为应对这些挑战，我们开发了PhishOracle，该工具通过将多样化的钓鱼特征嵌入合法网页来生成对抗性钓鱼网页。我们评估了两种现有模型（Stack模型和Phishpedia）在对PhishOracle生成的对抗性钓鱼网页进行分类时的鲁棒性。此外，我们研究了商用大型语言模型Gemini Pro Vision在对抗攻击背景下的表现。我们进行了一项用户研究，以确定PhishOracle生成的对抗性钓鱼网页是否能欺骗用户。我们的研究结果表明，许多PhishOracle生成的钓鱼网页能够规避当前的钓鱼网页检测模型并欺骗用户，但Gemini Pro Vision对该攻击表现出鲁棒性。我们还开发了PhishOracle网络应用程序，允许用户输入合法URL、选择相关钓鱼特征并生成相应的钓鱼网页。所有资源均在GitHub上公开提供。