With the increasing complexity and scope of software systems, their dependability is crucial. The analysis of log data recorded during system execution can enable engineers to automatically predict failures at run time. Several Machine Learning (ML) techniques, including traditional ML and Deep Learning (DL), have been proposed to automate such tasks. However, current empirical studies are limited in terms of covering all main DL types -- Recurrent Neural Network (RNN), Convolutional Neural network (CNN), and transformer -- as well as examining them on a wide range of diverse datasets. In this paper, we aim to address these issues by systematically investigating the combination of log data embedding strategies and DL types for failure prediction. To that end, we propose a modular architecture to accommodate various configurations of embedding strategies and DL-based encoders. To further investigate how dataset characteristics such as dataset size and failure percentage affect model accuracy, we synthesised 360 datasets, with varying characteristics, for three distinct system behavioral models, based on a systematic and automated generation approach. Using the F1 score metric, our results show that the best overall performing configuration is a CNN-based encoder with Logkey2vec. Additionally, we provide specific dataset conditions, namely a dataset size >350 or a failure percentage >7.5%, under which this configuration demonstrates high accuracy for failure prediction.

翻译：随着软件系统复杂性和规模的日益增长，其可靠性至关重要。通过分析系统执行期间记录的日志数据，工程师能够在运行时自动预测故障。多种机器学习技术，包括传统机器学习与深度学习，已被提出用于自动化此类任务。然而，当前实证研究在覆盖所有主要深度学习类型——循环神经网络、卷积神经网络和Transformer——以及在多样化数据集上进行检验方面存在局限性。本文旨在通过系统性地探索日志数据嵌入策略与深度学习类型在故障预测中的组合来应对这些问题。为此，我们提出了一种模块化架构，可适配多种嵌入策略与基于深度学习的编码器配置。为进一步研究数据集特征（如数据集规模和故障百分比）对模型准确率的影响，我们基于系统化的自动生成方法，合成了360个具有不同特征的数据集，这些数据来源于三种不同的系统行为模型。采用F1分数指标，我们的结果表明，整体性能最佳的配置是基于卷积神经网络的编码器结合Logkey2vec。此外，我们指出了该配置在故障预测中展现高准确率的具体数据集条件，即数据集规模大于350或故障百分比高于7.5%。