In recent years, spammers are now trying to obfuscate their intents by introducing hybrid spam e-mail combining both image and text parts, which is more challenging to detect in comparison to e-mails containing text or image only. The motivation behind this research is to design an effective approach filtering out hybrid spam e-mails to avoid situations where traditional text-based or image-baesd only filters fail to detect hybrid spam e-mails. To the best of our knowledge, a few studies have been conducted with the goal of detecting hybrid spam e-mails. Ordinarily, Optical Character Recognition (OCR) technology is used to eliminate the image parts of spam by transforming images into text. However, the research questions are that although OCR scanning is a very successful technique in processing text-and-image hybrid spam, it is not an effective solution for dealing with huge quantities due to the CPU power required and the execution time it takes to scan e-mail files. And the OCR techniques are not always reliable in the transformation processes. To address such problems, we propose new late multi-modal fusion training frameworks for a text-and-image hybrid spam e-mail filtering system compared to the classical early fusion detection frameworks based on the OCR method. Convolutional Neural Network (CNN) and Continuous Bag of Words were implemented to extract features from image and text parts of hybrid spam respectively, whereas generated features were fed to sigmoid layer and Machine Learning based classifiers including Random Forest (RF), Decision Tree (DT), Naive Bayes (NB) and Support Vector Machine (SVM) to determine the e-mail ham or spam.

翻译：近年来，垃圾邮件发送者试图通过引入包含图像和文本部分的混合垃圾邮件来混淆其意图，与仅包含文本或图像的邮件相比，这种邮件的检测更具挑战性。本研究的动机是设计一种有效的方法来过滤混合垃圾邮件，以避免传统的基于文本或基于图像的过滤器无法检测到混合垃圾邮件的情况。据我们所知，目前仅有少数研究致力于检测混合垃圾邮件。通常，光学字符识别（OCR）技术通过将图像转换为文本来消除垃圾邮件的图像部分。然而，研究问题在于，尽管OCR扫描在处理文本与图像混合垃圾邮件时是一种非常成功的技术，但由于处理邮件文件所需的CPU能力和执行时间，它并不是处理大量邮件的有效解决方案。此外，OCR技术在转换过程中并不总是可靠的。为了解决这些问题，我们提出了一种新的后期多模态融合训练框架，用于文本与图像混合垃圾邮件过滤系统，与基于OCR方法的经典早期融合检测框架相比。我们分别采用卷积神经网络（CNN）和连续词袋模型从混合垃圾邮件的图像和文本部分提取特征，而生成的特征被输入到sigmoid层以及基于机器学习的分类器，包括随机森林（RF）、决策树（DT）、朴素贝叶斯（NB）和支持向量机（SVM），以确定邮件是正常邮件还是垃圾邮件。