With the growth of online services, IoT devices, and DevOps-oriented software development, software log anomaly detection is becoming increasingly important. Prior works mainly follow a traditional four-staged architecture (Preprocessor, Parser, Vectorizer, and Classifier). This paper proposes OneLog, which utilizes a single Deep Neural Network (DNN) instead of multiple separate components. OneLog harnesses Convolutional Neural Networks (CNN) at the character level to take digits, numbers, and punctuations, which were removed in prior works, into account alongside the main natural language text. We evaluate our approach in six message- and sequence-based data sets: HDFS, Hadoop, BGL, Thunderbird, Spirit, and Liberty. We experiment with Onelog with single-, multi-, and cross-project setups. Onelog offers state-of-the-art performance in our datasets. Onelog can utilize multi-project datasets simultaneously during training, which suggests our model can generalize between datasets. Multi-project training also improves Onelog performance making it ideal when limited training data is available for an individual project. We also found that cross-project anomaly detection is possible with a single project pair (Liberty and Spirit). Analysis of model internals shows that one log has multiple modes of detecting anomalies and that the model learns manually validated parsing rules for the log messages. We conclude that character-based CNNs are a promising approach toward end-to-end learning in log anomaly detection. They offer good performance and generalization over multiple datasets. We will make our scripts publicly available upon the acceptance of this paper.

翻译：随着在线服务、物联网设备及DevOps导向软件开发的增长，软件日志异常检测变得日益重要。现有工作主要遵循传统的四阶段架构（预处理器、解析器、向量化器和分类器）。本文提出OneLog，它利用单一深度神经网络替代多个独立组件。OneLog在字符级采用卷积神经网络，将以往工作中被移除的数字、标点符号与主要自然语言文本一并纳入考量。我们在六个基于消息和序列的数据集（HDFS、Hadoop、BGL、Thunderbird、Spirit和Liberty）上评估了该方法，并在单项目、多项目及跨项目设置下进行实验。OneLog在数据集中实现了最先进的性能，且能在训练中同时利用多项目数据集，表明模型可跨数据集泛化。多项目训练还能提升OneLog性能，使其在单个项目训练数据有限时尤为理想。我们还发现，单个项目对（Liberty与Spirit）即可实现跨项目异常检测。模型内部机制分析表明，单个日志具有多种异常检测模式，且模型学习了经人工验证的日志消息解析规则。我们认为，基于字符的CNN是日志异常检测中实现端到端学习的前沿方向，能在多数据集上保持优异性能与泛化能力。本文录用后，我们将公开相关脚本。