StorSeismic is a recently introduced model based on the Transformer to adapt to various seismic processing tasks through its pretraining and fine-tuning training strategy. In the original implementation, StorSeismic utilized a sinusoidal positional encoding and a conventional self-attention mechanism, both borrowed from the natural language processing (NLP) applications. For seismic processing they admitted good results, but also hinted to limitations in efficiency and expressiveness. We propose modifications to these two key components, by utilizing relative positional encoding and low-rank attention matrices as replacements to the vanilla ones. The proposed changes are tested on processing tasks applied to a realistic Marmousi and offshore field data as a sequential strategy, starting from denoising, direct arrival removal, multiple attenuation, and finally root-mean-squared velocity ($V_{RMS}$) prediction for normal moveout (NMO) correction. We observe faster pretraining and competitive results on the fine-tuning tasks and, additionally, fewer parameters to train compared to the vanilla model.

翻译：StorSeismic是基于Transformer框架提出的新模型，通过预训练与微调策略可自适应多种地震处理任务。原始实现中，StorSeismic沿用了自然语言处理领域的正弦位置编码和常规自注意力机制。该方案在地震处理中虽取得良好效果，但也暴露出效率与表现力的局限性。本文针对这两个核心组件提出改进方案，采用相对位置编码和低秩注意力矩阵替代原始设计。我们将改进方法应用于真实Marmousi模型与海上油田数据的序列处理流程测试，依次完成去噪、直达波去除、多次波衰减，最终实现正常时差校正所需的均方根速度预测。实验表明，与原始模型相比，改进方案不仅预训练速度更快、微调任务效果更优，且参数量更少。