Smishing (SMS phishing) has become a serious cybersecurity threat, especially for elderly and cyber-unaware individuals, causing financial loss and undermining user trust. Although prior work has focused on detecting smishing at the level of individual messages, real-world attackers often rely on multi-stage social engineering, gradually manipulating victims through extended conversations before attempting to steal sensitive information. Despite the existence of several datasets for single-message smishing detection, datasets capturing conversational smishing remain largely unavailable, limiting research on multi-turn attack detection. To address this gap, this paper presents a synthetically generated dataset of 3,201 labeled multi-round conversations designed to emulate realistic conversational smishing attacks. The dataset reflects diverse attacker strategies and victim responses across multiple stages of interaction. Using this dataset, we establish baseline performance by evaluating eight models, including traditional machine learning approaches (Logistic Regression, Random Forest, Linear SVM, and XGBoost) and transformer-based architectures (DistilBERT and Longformer), with both engineered conversational features and TF-IDF text representations. Experimental results show that TF-IDF-based models consistently outperform those using engineered features alone. The best-performing model, XGBoost with TF-IDF features, achieves 72.5% accuracy and a macro F1 score of 0.691, surpassing both transformer models. Our analysis suggests that transformer performance is limited primarily by input-length constraints and the relatively small size of the training data. Overall, the results highlight the value of lexical signals in conversational smishing detection and demonstrate the usefulness of the proposed dataset for advancing research on defenses against multi-turn social engineering attacks.

翻译：短信钓鱼（SMS钓鱼）已成为一种严重的网络安全威胁，尤其对老年人和缺乏网络意识的群体而言，不仅造成经济损失，还削弱了用户信任。尽管以往研究侧重于单条消息层面的短信钓鱼检测，但现实攻击者往往依赖多阶段社会工程学手段，通过长对话逐步操控受害者，而后才试图窃取敏感信息。尽管存在几个用于单条短信钓鱼检测的数据集，但捕捉对话式短信钓鱼的数据集仍极为匮乏，限制了针对多轮攻击检测的研究。为填补这一空白，本文提出了一个包含3201条标注多轮对话的合成数据集，旨在模拟真实的对话式短信钓鱼攻击。该数据集反映了不同攻击策略及受害者在多阶段交互中的反应。利用该数据集，我们通过评估八种模型建立了基准性能，包括传统机器学习方法（逻辑回归、随机森林、线性支持向量机和XGBoost）和基于Transformer的架构（DistilBERT和Longformer），并结合了工程化的对话特征及TF-IDF文本表示。实验结果表明，基于TF-IDF的模型始终优于仅使用工程化特征的模型。性能最佳的模型——结合TF-IDF特征的XGBoost——取得了72.5%的准确率和0.691的宏F1分数，超越了两种Transformer模型。我们的分析表明，Transformer模型的性能主要受到输入长度限制及相对较小训练数据规模的制约。总体而言，这些结果突显了词汇信号在对话式短信钓鱼检测中的价值，并证明了所提数据集在推进针对多轮社会工程攻击防御研究方面的实用性。