With the rapid deployment of SCADA systems, how to effectively analyze industrial signals and detect abnormal states is an urgent need for the industry. Due to the significant heterogeneity of these signals, which we summarize as the M5 problem, previous works only focus on small sub-problems and employ specialized models, failing to utilize the synergies between modalities and the powerful scaling law. However, we argue that the M5 signals can be modeled in a unified manner due to the intrinsic similarity. As a result, we propose FISHER, a Foundation model for multi-modal Industrial Signal compreHEnsive Representation. To support arbitrary sampling rates, FISHER considers the increment of sampling rate as the concatenation of sub-band information. Specifically, FISHER takes the STFT sub-band as the modeling unit and adopts a teacher student SSL framework for pre-training. We also develop the RMIS benchmark, which evaluates the representations of M5 industrial signals on multiple health management tasks. Compared with top SSL models, FISHER showcases versatile and outstanding capabilities with a general performance gain up to 4.2%, along with much more efficient scaling curves. We also investigate the scaling law on downstream tasks and derive potential avenues for future work. Both FISHER and RMIS are now open-sourced.

翻译：随着SCADA系统的快速部署，如何有效分析工业信号并检测异常状态已成为工业界的迫切需求。由于这些信号存在显著的异质性（我们将其归纳为M5问题），先前研究仅关注局部子问题并采用专用模型，未能充分利用模态间的协同效应与强大的缩放定律。然而我们认为，M5信号因其内在相似性可采用统一方式建模。为此，我们提出FISHER——一种面向多模态工业信号综合表征的基础模型。为支持任意采样率，FISHER将采样率增量视为子带信息的级联。具体而言，FISHER以STFT子带作为建模单元，采用师生式自监督学习框架进行预训练。我们还开发了RMIS基准测试，通过在多个健康管理任务中评估M5工业信号的表征能力。与顶尖自监督模型相比，FISHER展现出全面且卓越的性能，综合性能提升最高达4.2%，并呈现更高效的缩放曲线。我们进一步探究了下游任务的缩放规律，为未来研究指明潜在方向。FISHER与RMIS现已开源。