Software systems often record important runtime information in logs to help with troubleshooting. Log-based anomaly detection has become a key research area that aims to identify system issues through log data, ultimately enhancing the reliability of software systems. Traditional deep learning methods often struggle to capture the semantic information embedded in log data, which is typically organized in natural language. In this paper, we propose LogLLM, a log-based anomaly detection framework that leverages large language models (LLMs). LogLLM employs BERT for extracting semantic vectors from log messages, while utilizing Llama, a transformer decoder-based model, for classifying log sequences. Additionally, we introduce a projector to align the vector representation spaces of BERT and Llama, ensuring a cohesive understanding of log semantics. Unlike conventional methods that require log parsers to extract templates, LogLLM preprocesses log messages with regular expressions, streamlining the entire process. Our framework is trained through a novel three-stage procedure designed to enhance performance and adaptability. Experimental results across four public datasets demonstrate that LogLLM outperforms state-of-the-art methods. Even when handling unstable logs, it effectively captures the semantic meaning of log messages and detects anomalies accurately.

翻译：软件系统通常将重要的运行时信息记录在日志中，以协助故障排查。基于日志的异常检测已成为一个关键的研究领域，旨在通过日志数据识别系统问题，最终提升软件系统的可靠性。传统的深度学习方法往往难以捕捉日志数据中蕴含的语义信息，这些信息通常以自然语言的形式组织。本文提出LogLLM，一个基于日志的异常检测框架，该框架利用大型语言模型（LLMs）。LogLLM采用BERT从日志消息中提取语义向量，同时利用基于Transformer解码器的模型Llama对日志序列进行分类。此外，我们引入了一个投影器来对齐BERT和Llama的向量表示空间，确保对日志语义的一致理解。与需要日志解析器提取模板的传统方法不同，LogLLM使用正则表达式对日志消息进行预处理，从而简化了整个流程。我们的框架通过一个新颖的三阶段训练过程进行训练，该过程旨在提升性能与适应性。在四个公开数据集上的实验结果表明，LogLLM优于现有最先进的方法。即使在处理不稳定的日志时，它也能有效捕捉日志消息的语义含义并准确检测异常。