Numerous Deep Learning (DL)-based approaches have gained attention in software Log Anomaly Detection (LAD), yet class imbalance in training data remains a challenge, with anomalies often comprising less than 1% of datasets like Thunderbird. Existing DLLAD methods may underperform in severely imbalanced datasets. Although data resampling has proven effective in other software engineering tasks, it has not been explored in LAD. This study aims to fill this gap by providing an in-depth analysis of the impact of diverse data resampling methods on existing DLLAD approaches from two distinct perspectives. Firstly, we assess the performance of these DLLAD approaches across four datasets with different levels of class imbalance, and we explore the impact of resampling ratios of normal to abnormal data on DLLAD approaches. Secondly, we evaluate the effectiveness of the data resampling methods when utilizing optimal resampling ratios of normal to abnormal data. Our findings indicate that oversampling methods generally outperform undersampling and hybrid sampling methods. Data resampling on raw data yields superior results compared to data resampling in the feature space. These improvements are attributed to the increased attention given to important tokens. By exploring the resampling ratio of normal to abnormal data, we suggest generating more data for minority classes through oversampling while removing less data from majority classes through undersampling. In conclusion, our study provides valuable insights into the intricate relationship between data resampling methods and DLLAD. By addressing the challenge of class imbalance, researchers and practitioners can enhance DLLAD performance.
翻译:在软件日志异常检测领域,众多基于深度学习的方法已受到广泛关注,但训练数据中的类别不平衡问题仍是严峻挑战——在诸如Thunderbird等数据集中,异常样本占比通常不足1%。现有的深度学习日志异常检测方法在严重不平衡数据集上可能表现不佳。尽管数据重采样技术在其他软件工程任务中已被证明有效,但在日志异常检测领域尚未得到充分探索。本研究旨在填补这一空白,从两个不同视角深入分析多种数据重采样方法对现有深度学习日志异常检测方法的影响。首先,我们在四个具有不同类别不平衡程度的数据集上评估这些深度学习日志异常检测方法的性能,并探究正常数据与异常数据的重采样比例对方法的影响。其次,我们评估了采用最优正常-异常数据重采样比例时,各数据重采样方法的有效性。研究结果表明:过采样方法通常优于欠采样和混合采样方法;相较于特征空间中的重采样,原始数据上的重采样能产生更优结果。这些改进归因于模型对重要标记的关注度提升。通过探索正常与异常数据的重采样比例,我们建议通过过采样为少数类生成更多数据,同时通过欠采样从多数类中移除较少数据。本研究最终揭示了数据重采样方法与深度学习日志异常检测之间复杂的相互作用关系。通过应对类别不平衡的挑战,研究人员与实践者能够有效提升深度学习日志异常检测的性能。