Artificial intelligence operations (AIOps) play a pivotal role in identifying, mitigating, and analyzing anomalous system behaviors and alerts. However, the research landscape in this field remains limited, leaving significant gaps unexplored. This study introduces a novel hybrid framework through an innovative algorithm that incorporates an unsupervised strategy. This strategy integrates Principal Component Analysis (PCA) and Artificial Neural Networks (ANNs) and uses a custom loss function to substantially enhance the effectiveness of log anomaly detection. The proposed approach encompasses the utilization of both simulated and real-world datasets, including logs from SockShop and Hadoop Distributed File System (HDFS). The experimental results are highly promising, demonstrating significant reductions in pseudo-positives. Moreover, this strategy offers notable advantages, such as the ability to process logs in their raw, unprocessed form, and the potential for further enhancements. The successful implementation of this approach showcases a remarkable reduction in anomalous logs, thus unequivocally establishing the efficacy of the proposed methodology. Ultimately, this study makes a substantial contribution to the advancement of log anomaly detection within AIOps platforms, addressing the critical need for effective and efficient log analysis in modern and complex systems.
翻译:人工智能运维(AIOps)在识别、缓解和分析异常系统行为及告警中发挥着关键作用。然而,该领域的研究格局仍存在局限,诸多重要空白有待探索。本研究通过一种融合无监督策略的创新算法,提出了一种新型混合框架。该策略整合主成分分析(PCA)与人工神经网络(ANN),并采用自定义损失函数,显著提升了日志异常检测的有效性。所提方法涵盖模拟数据集与真实世界数据集(包括SockShop及Hadoop分布式文件系统(HDFS)的日志)的运用。实验结果极具前景,伪阳性数量显著降低。此外,该策略具备突出优势,例如能够处理原始未处理日志,并具有进一步优化的潜力。该方法的成功实施展示了异常日志的显著减少,从而明确验证了所提方法的有效性。最终,本研究为AIOps平台中日志异常检测的进步做出了实质性贡献,有效满足了现代复杂系统中对高效日志分析的迫切需求。