Next activity prediction (NAP) is a cornerstone of predictive process monitoring (PPM), enabling organizations to move from retrospective analysis to proactive process steering. The PPM field has progressed from classical machine learning through deep learning architectures such as LSTMs and Transformers to large language models (LLMs). Despite growing model complexity, no benchmark jointly compares LLMs, Transformers, LSTMs, and simple baselines in a direct sequence modeling setting for NAP. In this paper, we fill this gap with a systematic benchmark. We compare vocabulary-adapted LLMs, Transformers trained from scratch, LLM-distilled Transformers, and LSTMs against a simple counting-based argmax baseline across seven real-life event logs. Our results tell a David vs. Goliath story: pretraining confers no consistent improvement over training from scratch, model size shows little effect on performance, and on most datasets the argmax baseline matches or approaches the performance of billion-parameter LLMs.

翻译：下一条活动预测（NAP）是预测性过程监控（PPM）的基石，使组织能够从回顾性分析转向主动式流程引导。PPM领域已从经典机器学习发展到LSTM和Transformer等深度学习架构，并进一步延伸至大语言模型（LLM）。尽管模型复杂度持续增长，目前尚无基准测试在直接序列建模场景下，联合比较LLM、Transformer、LSTM与简单基线方法在NAP任务上的表现。本文通过系统性基准测试填补了这一空白。我们在七个真实事件日志上，对比了词汇适配的LLM、从零训练的Transformer、经LLM蒸馏的Transformer以及LSTM与基于计数的简单Argmax基线。结果呈现了一个大卫与歌利亚式的故事：预训练并未持续优于从零训练，模型规模对性能影响甚微，且在多数数据集上，Argmax基线的表现能够匹配或接近十亿参数级LLM的水平。