With the increasing complexity and scope of software systems, their dependability is crucial. The analysis of log data recorded during system execution can enable engineers to automatically predict failures at run time. Several Machine Learning (ML) techniques, including traditional ML and Deep Learning (DL), have been proposed to automate such tasks. However, current empirical studies are limited in terms of covering all main DL types -- Recurrent Neural Network (RNN), Convolutional Neural network (CNN), and transformer -- as well as examining them on a wide range of diverse datasets. In this paper, we aim to address these issues by systematically investigating the combination of log data embedding strategies and DL types for failure prediction. To that end, we propose a modular architecture to accommodate various configurations of embedding strategies and DL-based encoders. To further investigate how dataset characteristics such as dataset size and failure percentage affect model accuracy, we synthesised 360 datasets, with varying characteristics, for three distinct system behavioral models, based on a systematic and automated generation approach. Using the F1 score metric, our results show that the best overall performing configuration is a CNN-based encoder with Logkey2vec. Additionally, we provide specific dataset conditions, namely a dataset size >350 or a failure percentage >7.5%, under which this configuration demonstrates high accuracy for failure prediction.

翻译：随着软件系统日益复杂和规模扩大，其可靠性至关重要。对系统运行时记录的日志数据进行分析，可使工程师在运行时自动预测故障。目前已提出多种机器学习（ML）技术（包括传统ML和深度学习（DL））来自动化此类任务。然而，当前实证研究在覆盖所有主要DL类型——循环神经网络（RNN）、卷积神经网络（CNN）和Transformer——以及基于多样化数据集进行检验方面存在局限。本文旨在通过系统研究日志数据嵌入策略与DL类型在故障预测中的组合来应对这些问题。为此，我们提出了一种模块化架构，以兼容嵌入策略和基于DL的编码器的各种配置。为进一步探究数据集规模与故障比例等特征如何影响模型准确性，我们基于系统化自动生成方法，针对三种不同的系统行为模型合成了360个具有不同特征的数据集。使用F1分数指标，我们的结果表明整体性能最佳的配置是采用Logkey2vec的CNN编码器。此外，我们明确了特定数据集条件（即数据集规模>350或故障比例>7.5%），在此条件下该配置能实现高精度的故障预测。