We propose BlockFound, a customized foundation model for anomaly blockchain transaction detection. Unlike existing methods that rely on rule-based systems or directly apply off-the-shelf large language models, BlockFound introduces a series of customized designs to model the unique data structure of blockchain transactions. First, a blockchain transaction is multi-modal, containing blockchain-specific tokens, texts, and numbers. We design a modularized tokenizer to handle these multi-modal inputs, balancing the information across different modalities. Second, we design a customized mask language learning mechanism for pretraining with RoPE embedding and FlashAttention for handling longer sequences. After training the foundation model, we further design a novel detection method for anomaly detection. Extensive evaluations on Ethereum and Solana transactions demonstrate BlockFound's exceptional capability in anomaly detection while maintaining a low false positive rate. Remarkably, BlockFound is the only method that successfully detects anomalous transactions on Solana with high accuracy, whereas all other approaches achieved very low or zero detection recall scores. This work not only provides new foundation models for blockchain but also sets a new benchmark for applying LLMs in blockchain data.

翻译：本文提出BlockFound，一种用于区块链异常交易检测的定制化基础模型。与现有依赖规则系统或直接应用现成大型语言模型的方法不同，BlockFound引入了一系列定制化设计，以建模区块链交易特有的数据结构。首先，区块链交易具有多模态特性，包含区块链特有的代币、文本和数值。我们设计了一个模块化分词器来处理这些多模态输入，平衡不同模态间的信息。其次，我们设计了一种结合RoPE嵌入与FlashAttention的定制化掩码语言学习机制进行预训练，以处理更长的序列。在训练基础模型后，我们进一步设计了一种新颖的异常检测方法。在以太坊和Solana交易数据上的大量评估表明，BlockFound在保持低误报率的同时，展现出卓越的异常检测能力。值得注意的是，BlockFound是唯一能在Solana上以高准确率成功检测异常交易的方法，而其他所有方法的检测召回率均极低或为零。这项工作不仅为区块链领域提供了新的基础模型，也为在区块链数据中应用大语言模型设立了新的基准。