Logs have been widely adopted in software system development and maintenance because of the rich runtime information they record. In recent years, the increase of software size and complexity leads to the rapid growth of the volume of logs. To handle these large volumes of logs efficiently and effectively, a line of research focuses on developing intelligent and automated log analysis techniques. However, only a few of these techniques have reached successful deployments in industry due to the lack of public log datasets and open benchmarking upon them. To fill this significant gap and facilitate more research on AI-driven log analytics, we have collected and released loghub, a large collection of system log datasets. In particular, loghub provides 19 real-world log datasets collected from a wide range of software systems, including distributed systems, supercomputers, operating systems, mobile systems, server applications, and standalone software. In this paper, we summarize the statistics of these datasets, introduce some practical usage scenarios of the loghub datasets, and present our benchmarking results on loghub to benefit the researchers and practitioners in this field. Up to the time of this paper writing, the loghub datasets have been downloaded for roughly 90,000 times in total by hundreds of organizations from both industry and academia. The loghub datasets are available at https://github.com/logpai/loghub.

翻译：日志因记录丰富的运行时信息，已被广泛应用于软件系统的开发与维护。近年来，软件规模与复杂度的提升导致日志量快速增长。为高效处理海量日志，相关研究聚焦于开发智能自动化日志分析技术。然而，由于缺乏公开日志数据集及基于这些数据集的开放式基准测试，实际工业部署的这些技术寥寥可数。为填补这一重大空白并促进更多AI驱动日志分析研究，我们收集并发布了loghub——一个大型系统日志数据集合集。具体而言，loghub提供了19个来自分布式系统、超级计算机、操作系统、移动系统、服务器应用及独立软件等广泛软件系统的真实日志数据集。本文汇总了这些数据集的统计特征，介绍了loghub数据集的实际应用场景，并展示了基于loghub的基准测试结果，以惠及该领域的研究者与实践者。截至本文撰写时，loghub数据集已被工业界与学术界的数百个组织累计下载约9万次。loghub数据集可通过https://github.com/logpai/loghub获取。