Multivariate time-series data in numerous real-world applications (e.g., healthcare and industry) are informative but challenging due to the lack of labels and high dimensionality. Recent studies in self-supervised learning have shown their potential in learning rich representations without relying on labels, yet they fall short in learning disentangled embeddings and addressing issues of inductive bias (e.g., transformation-invariance). To tackle these challenges, we propose TimeDRL, a generic multivariate time-series representation learning framework with disentangled dual-level embeddings. TimeDRL is characterized by three novel features: (i) disentangled derivation of timestamp-level and instance-level embeddings from patched time-series data using a [CLS] token strategy; (ii) utilization of timestamp-predictive and instance-contrastive tasks for disentangled representation learning, with the former optimizing timestamp-level embeddings with predictive loss, and the latter optimizing instance-level embeddings with contrastive loss; and (iii) avoidance of augmentation methods to eliminate inductive biases, such as transformation-invariance from cropping and masking. Comprehensive experiments on 6 time-series forecasting datasets and 5 time-series classification datasets have shown that TimeDRL consistently surpasses existing representation learning approaches, achieving an average improvement of forecasting by 58.02% in MSE and classification by 1.48% in accuracy. Furthermore, extensive ablation studies confirmed the relative contribution of each component in TimeDRL's architecture, and semi-supervised learning evaluations demonstrated its effectiveness in real-world scenarios, even with limited labeled data. The code is available at https://github.com/blacksnail789521/TimeDRL.

翻译：多变量时间序列数据在诸多实际应用（如医疗和工业）中信息丰富，但因缺乏标签和高维度特性而极具挑战性。近期自监督学习研究虽展示了无需依赖标签即可学习丰富表征的潜力，但在学习解耦嵌入及处理归纳偏差（如变换不变性）方面仍存在不足。为解决这些挑战，我们提出TimeDRL——一种具有解耦双层嵌入的通用多变量时间序列表示学习框架。TimeDRL具备三大创新特征：（i）利用[CLS]标记策略，从分块时间序列数据中解耦推导时间戳级和实例级嵌入；（ii）采用时间戳预测任务与实例对比任务进行解耦表示学习，前者通过预测损失优化时间戳级嵌入，后者通过对比损失优化实例级嵌入；（iii）避免使用数据增强方法以消除归纳偏差（如裁剪和掩码导致的变换不变性）。在6个时间序列预测数据集和5个时间序列分类数据集上的全面实验表明，TimeDRL始终超越现有表示学习方法，在预测任务上均方误差平均降低58.02%，在分类任务上准确率平均提升1.48%。此外，大量消融实验验证了TimeDRL架构中各组件的相对贡献，半监督学习评估则证明了其在标注数据有限的真实场景中的有效性。代码已开源至https://github.com/blacksnail789521/TimeDRL。