The growth of global consumption has motivated important applications of deep learning to smart manufacturing and machine health monitoring. In particular, analyzing vibration data offers great potential to extract meaningful insights into predictive maintenance by the detection of bearing faults. Deep learning can be a powerful method to predict these mechanical failures; however, they lack generalizability to new tasks or datasets and require expensive, labeled mechanical data. We address this by presenting a novel self-supervised pretraining and fine-tuning framework based on transformer models. In particular, we investigate different tokenization and data augmentation strategies to reach state-of-the-art accuracies using transformer models. Furthermore, we demonstrate self-supervised masked pretraining for vibration signals and its application to low-data regimes, task adaptation, and dataset adaptation. Pretraining is able to improve performance on scarce, unseen training samples, as well as when fine-tuning on fault classes outside of the pretraining distribution. Furthermore, pretrained transformers are shown to be able to generalize to a different dataset in a few-shot manner. This introduces a new paradigm where models can be pretrained on unlabeled data from different bearings, faults, and machinery and quickly deployed to new, data-scarce applications to suit specific manufacturing needs.

翻译：全球消费的增长推动了深度学习在智能制造与机器健康监测中的重要应用。具体而言，振动数据分析为通过轴承故障检测实现预测性维护提供了提取有意义洞察的巨大潜力。深度学习可以成为预测这些机械故障的有力方法；然而，现有方法缺乏对新任务或数据集的泛化能力，且需要昂贵且带标签的机械数据。为此，我们提出了一种基于Transformer模型的新型自监督预训练与微调框架。我们特别研究了不同的分词与数据增强策略，以利用Transformer模型达到最先进的准确率。此外，我们展示了针对振动信号的自监督掩码预训练及其在低数据场景、任务适应与数据集适应中的应用。预训练能够提升在稀缺、未见过的训练样本上的性能，以及在预训练分布之外的故障类别上进行微调时的表现。此外，研究表明预训练的Transformer能够以少量样本的方式泛化到不同的数据集。这引入了一种新范式：模型可以在来自不同轴承、故障类型和机械设备的无标签数据上进行预训练，并快速部署到新的、数据稀缺的应用中，以满足特定的制造需求。