In recent years, spammers are now trying to obfuscate their intents by introducing hybrid spam e-mail combining both image and text parts, which is more challenging to detect in comparison to e-mails containing text or image only. The motivation behind this research is to design an effective approach filtering out hybrid spam e-mails to avoid situations where traditional text-based or image-baesd only filters fail to detect hybrid spam e-mails. To the best of our knowledge, a few studies have been conducted with the goal of detecting hybrid spam e-mails. Ordinarily, Optical Character Recognition (OCR) technology is used to eliminate the image parts of spam by transforming images into text. However, the research questions are that although OCR scanning is a very successful technique in processing text-and-image hybrid spam, it is not an effective solution for dealing with huge quantities due to the CPU power required and the execution time it takes to scan e-mail files. And the OCR techniques are not always reliable in the transformation processes. To address such problems, we propose new late multi-modal fusion training frameworks for a text-and-image hybrid spam e-mail filtering system compared to the classical early fusion detection frameworks based on the OCR method. Convolutional Neural Network (CNN) and Continuous Bag of Words were implemented to extract features from image and text parts of hybrid spam respectively, whereas generated features were fed to sigmoid layer and Machine Learning based classifiers including Random Forest (RF), Decision Tree (DT), Naive Bayes (NB) and Support Vector Machine (SVM) to determine the e-mail ham or spam.

翻译：近年来，垃圾邮件发送者试图通过引入包含图像和文本部分的混合垃圾邮件来混淆其意图，与仅包含文本或图像的邮件相比，这种邮件检测更具挑战性。本研究旨在设计一种有效过滤混合垃圾邮件的方法，以避免传统基于文本或图像的过滤器无法检测混合垃圾邮件的情况。据我们所知，目前针对混合垃圾邮件检测的研究较少。通常，光学字符识别（OCR）技术被用于将图像转换为文本以消除垃圾邮件的图像部分。然而，研究问题在于，尽管OCR扫描在处理文本-图像混合垃圾邮件时是一种非常成功的技术，但由于扫描邮件文件所需的CPU算力和执行时间，它并非处理大量邮件的有效解决方案，且OCR技术在转换过程中并非始终可靠。为解决这些问题，我们提出了一种新的后期多模态融合训练框架，用于文本-图像混合垃圾邮件过滤系统，与基于OCR方法的经典早期融合检测框架形成对比。我们分别采用卷积神经网络（CNN）和连续词袋模型从混合垃圾邮件的图像和文本部分提取特征，并将生成的特征输入sigmoid层及基于机器学习的分类器，包括随机森林（RF）、决策树（DT）、朴素贝叶斯（NB）和支持向量机（SVM），以判断邮件是否为正常邮件或垃圾邮件。