Modern financial systems generate vast quantities of transactional and event-level data that encode rich economic signals. This paper presents PRAGMA, a family of foundation models for multi-source banking event sequences. Our approach pre-trains a Transformer-based architecture with masked modelling on a large-scale, heterogeneous banking event corpus using a self-supervised objective tailored to the discrete, variable-length nature of financial records. The resulting model supports a wide range of downstream tasks such as credit scoring, fraud detection, and lifetime value prediction: strong performance can be achieved by training a simple linear model on top of the extracted embeddings and can be further improved with lightweight fine-tuning. Through extensive evaluation on downstream tasks, we demonstrate that PRAGMA achieves superior performance across multiple domains directly from raw event sequences, providing a general-purpose representation layer for financial applications.

翻译：[translated abstract in Chinese] 现代金融系统产生了海量的交易级与事件级数据，这些数据编码了丰富的经济信号。本文提出PRAGMA——面向多源银行事件序列的基础模型系列。该方法采用基于Transformer的架构，结合针对金融记录离散、变长特性设计的自监督目标，在大规模异构银行事件语料上进行掩码建模预训练。所得模型支持信用评分、欺诈检测与生命周期价值预测等多种下游任务：仅需在提取的嵌入表示上训练简单线性模型即可获得优异性能，通过轻量级微调可进一步提升效果。基于下游任务的广泛评估表明，PRAGMA能够直接从原始事件序列中实现跨多个领域的卓越性能，为金融应用提供了通用化表征层。