The emergence of abundant electricity time series (ETS) data provides ample opportunities for various applications in the power systems, including demand-side management, grid stability, and consumer behavior analysis. Deep learning models have advanced ETS modeling by effectively capturing sequence dependence. Nevertheless, learning a generic representation of ETS data for various applications remains challenging due to the inherently complex hierarchical structure of ETS data. Moreover, ETS data exhibits intricate temporal dependencies and is suscepti ble to the influence of exogenous variables. Furthermore, different instances exhibit diverse electricity consumption behavior. In this paper, we propose a foundation model PowerPM to model ETS data, providing a large-scale, off-the-shelf model for power systems. PowerPM consists of a temporal encoder and a hierarchical encoder. The temporal encoder captures both temporal dependencies in ETS data, considering exogenous variables. The hierarchical encoder models the correlation between hierarchy. Furthermore, PowerPM leverages a novel self-supervised pretraining framework consisting of masked ETS modeling and dual-view contrastive learning, which enable PowerPM to capture temporal dependency within ETS windows and aware the discrepancy across ETS windows, providing two different perspectives to learn generic representation. Our experiments involve five real world scenario datasets, comprising private and public data. Through pre-training on massive ETS data, PowerPM achieves SOTA performance on diverse downstream tasks within the private dataset. Impressively, when transferred to the public datasets, PowerPM maintains its superiority, showcasing its remarkable generalization ability across various tasks and domains. Moreover, ablation studies, few-shot experiments provide additional evidence of the effectiveness of our model.

翻译：电力时间序列（ETS）数据的涌现为电力系统中的多种应用提供了丰富机遇，包括需求侧管理、电网稳定性和用户行为分析。深度学习模型通过有效捕捉序列依赖性，推动了ETS建模的发展。然而，由于ETS数据固有的复杂层次结构，学习适用于多种应用的通用ETS表示仍具挑战性。此外，ETS数据展现出复杂的时间依赖性，且易受外生变量影响。同时，不同实例的用电行为呈现显著差异。本文提出基础模型PowerPM对ETS数据进行建模，为电力系统提供大规模即用型模型。PowerPM由时间编码器与层次编码器构成：时间编码器在考虑外生变量的同时捕捉ETS数据中的时间依赖性；层次编码器则建模层次间的关联关系。进一步地，PowerPM采用包含掩码ETS建模与双视图对比学习的新型自监督预训练框架，使其既能捕捉ETS窗口内的时间依赖性，又能感知ETS窗口间的差异性，从而通过双重视角学习通用表示。我们在包含私有与公开数据的五个真实场景数据集上进行实验。通过对海量ETS数据进行预训练，PowerPM在私有数据集内的多项下游任务中达到最先进性能。值得注意的是，当迁移至公开数据集时，PowerPM仍保持其优越性，展现出跨任务与跨领域的卓越泛化能力。此外，消融研究与少样本实验为模型有效性提供了进一步佐证。