Log Anomaly Detection (LAD) seeks to identify atypical patterns in log data that are crucial to assessing the security and condition of systems. Although Large Language Models (LLMs) have shown tremendous success in various fields, the use of LLMs in enabling the detection of log anomalies is largely unexplored. This work aims to fill this gap. Due to the prohibitive costs involved in fully fine-tuning LLMs, we explore the use of parameter-efficient fine-tuning techniques (PEFTs) for adapting LLMs to LAD. To have an in-depth exploration of the potential of LLM-driven LAD, we present a comprehensive investigation of leveraging two of the most popular PEFTs -- Low-Rank Adaptation (LoRA) and Representation Fine-tuning (ReFT) -- to tap into three prominent LLMs of varying size, including RoBERTa, GPT-2, and Llama-3, for parameter-efficient LAD. Comprehensive experiments on four public log datasets are performed to reveal important insights into effective LLM-driven LAD in several key perspectives, including the efficacy of these PEFT-based LLM-driven LAD methods, their stability, sample efficiency, robustness w.r.t. unstable logs, and cross-dataset generalization. Code is available at https://github.com/mala-lab/LogADReft.

翻译：日志异常检测（Log Anomaly Detection, LAD）旨在识别日志数据中的异常模式，这对于评估系统的安全性和状态至关重要。尽管大语言模型（Large Language Models, LLMs）已在多个领域取得巨大成功，但利用LLMs实现日志异常检测的研究在很大程度上仍未得到探索。本研究旨在填补这一空白。由于对LLMs进行全参数微调的成本过高，我们探索使用参数高效微调技术（Parameter-Efficient Fine-Tuning techniques, PEFTs）来使LLMs适应LAD任务。为了深入探索LLM驱动的LAD的潜力，我们进行了全面的研究，利用两种最流行的PEFT方法——低秩适应（Low-Rank Adaptation, LoRA）和表征微调（Representation Fine-tuning, ReFT）——来挖掘三个不同规模的知名LLM（包括RoBERTa、GPT-2和Llama-3）在参数高效LAD方面的能力。我们在四个公开日志数据集上进行了全面的实验，从多个关键视角揭示了有效LLM驱动LAD的重要见解，包括这些基于PEFT的LLM驱动LAD方法的有效性、稳定性、样本效率、对不稳定日志的鲁棒性以及跨数据集泛化能力。代码可在 https://github.com/mala-lab/LogADReft 获取。