Log parsing, which involves log template extraction from semi-structured logs to produce structured logs, is the first and the most critical step in automated log analysis. However, current log parsers suffer from limited effectiveness for two reasons. First, traditional data-driven log parsers solely rely on heuristics or handcrafted features designed by domain experts, which may not consistently perform well on logs from diverse systems. Second, existing supervised log parsers require model tuning, which is often limited to fixed training samples and causes sub-optimal performance across the entire log source. To address this limitation, we propose DivLog, an effective log parsing framework based on the in-context learning (ICL) ability of large language models (LLMs). Specifically, before log parsing, DivLog samples a small amount of offline logs as candidates by maximizing their diversity. Then, during log parsing, DivLog selects five appropriate labeled candidates as examples for each target log and constructs them into a prompt. By mining the semantics of examples in the prompt, DivLog generates a target log template in a training-free manner. In addition, we design a straightforward yet effective prompt format to extract the output and enhance the quality of the generated log templates. We conducted experiments on 16 widely-used public datasets. The results show that DivLog achieves (1) 98.1% Parsing Accuracy, (2) 92.1% Precision Template Accuracy, and (3) 92.9% Recall Template Accuracy on average, exhibiting state-of-the-art performance.

翻译：日志解析，即从半结构化日志中提取日志模板以生成结构化日志，是自动化日志分析中首要且最关键的步骤。然而，当前日志解析器的有效性受限于两个原因：第一，传统数据驱动型日志解析器仅依赖领域专家设计的启发式规则或手工特征，在处理多种系统日志时难以保持稳定性能；第二，现有监督式日志解析器需进行模型调优，但常局限于固定训练样本，导致对整个日志源的解析性能次优。为解决该问题，我们提出DivLog——一种基于大语言模型（LLMs）上下文学习（ICL）能力的高效日志解析框架。具体而言，在日志解析前，DivLog通过最大化多样性策略从离线日志中采样少量候选样本；日志解析阶段，DivLog为每条目标日志选取五个带标签的候选样本作为示例，并构建为提示（prompt）。通过挖掘提示中示例的语义信息，DivLog以无需训练的方式生成目标日志模板。此外，我们设计了简洁高效的提示格式以提取输出并提升生成日志模板的质量。在16个广泛使用的公开数据集上的实验表明，DivLog平均达到：（1）98.1%的解析精度（Parsing Accuracy）、（2）92.1%的精确模板准确率（Precision Template Accuracy）及（3）92.9%的召回模板准确率（Recall Template Accuracy），展现出最优性能。