Recent advancements in foundation models have been successfully extended to the time series (TS) domain, facilitated by the emergence of large-scale TS datasets. However, previous efforts have primarily Capturing channel dependency (CD) is essential for modeling multivariate time series (TS), and attention-based methods have been widely employed for this purpose. Nonetheless, these methods primarily focus on modifying the architecture, often neglecting the importance of dataset-specific characteristics. In this work, we introduce the concept of partial channel dependence (PCD) to enhance CD modeling in Transformer-based models by leveraging dataset-specific information to refine the CD captured by the model. To achieve PCD, we propose channel masks (CMs), which are integrated into the attention matrices of Transformers via element-wise multiplication. CMs consist of two components: 1) a similarity matrix that captures relationships between the channels, and 2) dataset-specific and learnable domain parameters that refine the similarity matrix. We validate the effectiveness of PCD across diverse tasks and datasets with various backbones. Code is available at this repository: https://github.com/YonseiML/pcd.

翻译：近年来，基础模型的进展已成功扩展至时间序列领域，这得益于大规模时间序列数据集的涌现。然而，先前的研究主要聚焦于通过调整架构来捕捉通道依赖性，而通道依赖性的建模对于多元时间序列分析至关重要，基于注意力机制的方法已被广泛用于此目的。尽管如此，这些方法往往忽视了数据集特定特征的重要性。在本研究中，我们引入了部分通道依赖性的概念，旨在通过利用数据集特定信息来优化Transformer模型所捕获的通道依赖性，从而增强基于Transformer的模型在通道依赖性建模方面的性能。为实现部分通道依赖性，我们提出了通道掩码，该掩码通过逐元素乘法集成到Transformer的注意力矩阵中。通道掩码由两个部分组成：1) 捕捉通道间关系的相似性矩阵，以及2) 基于数据集特定且可学习的领域参数，用于优化相似性矩阵。我们在多种任务和数据集上，结合不同的骨干网络，验证了部分通道依赖性的有效性。代码可在以下仓库获取：https://github.com/YonseiML/pcd。