This paper introduces TRACE-GPT, which stands for Time-seRies Anomaly-detection with Convolutional Embedding and Generative Pre-trained Transformers. TRACE-GPT is designed to pre-train univariate time-series sensor data and detect faults on unlabeled datasets in semiconductor manufacturing. In semiconductor industry, classifying abnormal time-series sensor data from normal data is important because it is directly related to wafer defect. However, small, unlabeled, and even mixed training data without enough anomalies make classification tasks difficult. In this research, we capture features of time-series data with temporal convolutional embedding and Generative Pre-trained Transformer (GPT) to classify abnormal sequences from normal sequences using cross entropy loss. We prove that our model shows better performance than previous unsupervised models with both an open dataset, the University of California Riverside (UCR) time-series classification archive, and the process log of our Chemical Vapor Deposition (CVD) equipment. Our model has the highest F1 score at Equal Error Rate (EER) across all datasets and is only 0.026 below the supervised state-of-the-art baseline on the open dataset.

翻译：本文介绍了TRACE-GPT，即基于卷积嵌入与生成式预训练Transformer的时间序列异常检测模型。TRACE-GPT旨在对单变量时间序列传感器数据进行预训练，并在无标签数据集上检测半导体制造中的故障。在半导体行业中，将异常时间序列传感器数据与正常数据分类至关重要，因为这直接关系到晶圆缺陷。然而，训练数据规模小、无标签甚至混合数据中缺乏足够异常样本等问题，使得分类任务颇具挑战。在本研究中，我们利用时序卷积嵌入与生成式预训练Transformer（GPT）捕捉时间序列数据特征，通过交叉熵损失对异常序列与正常序列进行分类。我们证明，该模型在开放数据集——加州大学河滨分校（UCR）时间序列分类数据库——以及我们化学气相沉积（CVD）设备的工艺日志上，均表现出优于先前无监督模型的性能。我们的模型在所有数据集上均获得等错误率（EER）下的最高F1分数，且在开放数据集上仅比有监督的最先进基线低0.026。