High-quality datasets are critical for training machine learning models, as inconsistencies in feature generation can hinder the accuracy and reliability of threat detection. For this reason, ensuring the quality of the data in network intrusion detection datasets is important. A key component of this is using reliable tools to generate the flows and features present in the datasets. This paper investigates the impact of flow exporters on the performance and reliability of machine learning models for intrusion detection. Using HERA, a tool designed to export flows and extract features, the raw network packets of two widely used datasets, UNSW-NB15 and CIC-IDS2017, were processed from PCAP files to generate new versions of these datasets. These were compared to the original ones in terms of their influence on the performance of several models, including Random Forest, XGBoost, LightGBM, and Explainable Boosting Machine. The results obtained were significant. Models trained on the HERA version of the datasets consistently outperformed those trained on the original dataset, showing improvements in accuracy and indicating a better generalisation. This highlighted the importance of flow generation in the model's ability to differentiate between benign and malicious traffic.

翻译：高质量数据集对于训练机器学习模型至关重要，因为特征生成过程中的不一致性会阻碍威胁检测的准确性和可靠性。因此，确保网络入侵检测数据集中数据的质量非常重要。其中一个关键环节是使用可靠的工具来生成数据集中存在的流和特征。本文研究了流导出器对用于入侵检测的机器学习模型的性能和可靠性的影响。使用专为导出流和提取特征而设计的工具HERA，对两个广泛使用的数据集（UNSW-NB15和CIC-IDS2017）的原始网络数据包进行处理，从PCAP文件生成这些数据集的新版本。这些新版本与原始版本在影响多种模型（包括随机森林、XGBoost、LightGBM和可解释提升机）性能方面进行了比较。获得的结果具有显著性。在数据集的HERA版本上训练的模型，其性能始终优于在原始数据集上训练的模型，显示出准确性的提升，并表明其具有更好的泛化能力。这突显了流生成对于模型区分良性流量和恶意流量能力的重要性。