The process mining community has recently recognized the potential of large language models (LLMs) for tackling various process mining tasks. Initial studies report the capability of LLMs to support process analysis and even, to some extent, that they are able to reason about how processes work. This latter property suggests that LLMs could also be used to tackle process mining tasks that benefit from an understanding of process behavior. Examples of such tasks include (semantic) anomaly detection and next activity prediction, which both involve considerations of the meaning of activities and their inter-relations. In this paper, we investigate the capabilities of LLMs to tackle such semantics-aware process mining tasks. Furthermore, whereas most works on the intersection of LLMs and process mining only focus on testing these models out of the box, we provide a more principled investigation of the utility of LLMs for process mining, including their ability to obtain process mining knowledge post-hoc by means of in-context learning and supervised fine-tuning. Concretely, we define three process mining tasks that benefit from an understanding of process semantics and provide extensive benchmarking datasets for each of them. Our evaluation experiments reveal that (1) LLMs fail to solve challenging process mining tasks out of the box and when provided only a handful of in-context examples, (2) but they yield strong performance when fine-tuned for these tasks, consistently surpassing smaller, encoder-based language models.

翻译：过程挖掘领域近期认识到大语言模型（LLMs）在处理各类过程挖掘任务方面的潜力。初步研究表明，大语言模型能够支持过程分析，甚至在一定程度上能够推理过程的运作机制。后一特性表明，大语言模型也可用于处理那些受益于理解过程行为的过程挖掘任务，例如（语义）异常检测和下一活动预测——这两类任务均涉及对活动含义及其相互关系的考量。本文系统研究了大语言模型处理此类语义感知过程挖掘任务的能力。此外，现有关于大语言模型与过程挖掘交叉领域的研究大多仅测试模型的即用性能，而本文通过情境学习和监督微调等方式，对模型获取过程挖掘知识的能力进行了更具原则性的探究。具体而言，我们定义了三种受益于过程语义理解的过程挖掘任务，并为每项任务提供了详尽的基准数据集。评估实验表明：（1）大语言模型在即用状态下及仅提供少量情境示例时，难以解决具有挑战性的过程挖掘任务；（2）但经过任务特定微调后，其表现显著提升，持续超越基于编码器的小型语言模型。