Dynamic analysis enables detecting Windows malware by executing programs in a controlled environment and logging their actions. Previous work has proposed training machine learning models, i.e., convolutional and long short-term memory networks, on homogeneous input features like runtime APIs to either detect or classify malware, neglecting other relevant information coming from heterogeneous data like network and file operations. To overcome these issues, we introduce Nebula, a versatile, self-attention Transformer-based neural architecture that generalizes across different behavioral representations and formats, combining diverse information from dynamic log reports. Nebula is composed by several components needed to tokenize, filter, normalize and encode data to feed the transformer architecture. We firstly perform a comprehensive ablation study to evaluate their impact on the performance of the whole system, highlighting which components can be used as-is, and which must be enriched with specific domain knowledge. We perform extensive experiments on both malware detection and classification tasks, using three datasets acquired from different dynamic analyses platforms, show that, on average, Nebula outperforms state-of-the-art models at low false positive rates, with a peak of 12% improvement. Moreover, we showcase how self-supervised learning pre-training matches the performance of fully-supervised models with only 20% of training data, and we inspect the output of Nebula through explainable AI techniques, pinpointing how attention is focusing on specific tokens correlated to malicious activities of malware families. To foster reproducibility, we open-source our findings and models at https://github.com/dtrizna/nebula.

翻译：动态分析通过在受控环境中执行程序并记录其行为，实现对Windows恶意软件的检测。先前研究提出基于运行时API等同质输入特征训练机器学习模型（如卷积神经网络和长短期记忆网络）进行恶意软件检测或分类，但忽略了来自网络和文件操作等异构数据的其他相关信息。为克服这些问题，我们提出星云（Nebula）——一种基于自注意力Transformer架构的通用神经网络，能够泛化处理不同行为表征与数据格式，整合动态日志报告中多样化的信息。星云包含多个组件，用于对数据进行分词、过滤、归一化和编码，以输入Transformer架构。我们首先通过全面的消融实验评估各组件对整个系统性能的影响，明确哪些组件可直接使用，哪些需融入特定领域知识进行增强。我们在三个来自不同动态分析平台的数据集上，对恶意软件检测与分类任务进行了大量实验，结果表明星云在低误报率条件下平均性能优于现有最优模型，最高提升达12%。此外，我们展示了自监督学习预训练仅需20%训练数据即可达到全监督模型的性能，并通过可解释AI技术解析星云的输出，揭示注意力机制如何聚焦于与恶意软件家族恶意活动相关的特定标记。为促进可复现性，我们在https://github.com/dtrizna/nebula开源了研究成果与模型。