Multi-dimensional time series data, such as matrix and tensor-variate time series, are increasingly prevalent in fields such as economics, finance, and climate science. Traditional Transformer models, though adept with sequential data, do not effectively preserve these multi-dimensional structures, as their internal operations in effect flatten multi-dimensional observations into vectors, thereby losing critical multi-dimensional relationships and patterns. To address this, we introduce the Tensor-Augmented Transformer (TEAFormer), a novel method that incorporates tensor expansion and compression within the Transformer framework to maintain and leverage the inherent multi-dimensional structures, thus reducing computational costs and improving prediction accuracy. The core feature of the TEAFormer, the Tensor-Augmentation (TEA) module, utilizes tensor expansion to enhance multi-view feature learning and tensor compression for efficient information aggregation and reduced computational load. The TEA module is not just a specific model architecture but a versatile component that is highly compatible with the attention mechanism and the encoder-decoder structure of Transformers, making it adaptable to existing Transformer architectures. Our comprehensive experiments, which integrate the TEA module into three popular time series Transformer models across three real-world benchmarks, show significant performance enhancements, highlighting the potential of TEAFormers for cutting-edge time series forecasting.

翻译：多维时间序列数据（如矩阵和张量时间序列）在经济学、金融学和气候科学等领域日益普遍。传统的Transformer模型虽然擅长处理序列数据，但无法有效保留这些多维结构，因为其内部操作实际上将多维观测值展平为向量，从而丢失了关键的多维关系和模式。为解决这一问题，我们提出了张量增强Transformer（TEAFormer），这是一种在Transformer框架内结合张量扩展与压缩的新方法，旨在保持并利用固有的多维结构，从而降低计算成本并提高预测精度。TEAFormer的核心特征——张量增强（TEA）模块，利用张量扩展来增强多视图特征学习，并通过张量压缩实现高效的信息聚合与计算负载降低。TEA模块不仅是一种特定的模型架构，更是一个高度兼容Transformer注意力机制和编码器-解码器结构的通用组件，使其能够适配现有的Transformer架构。我们通过将TEA模块集成到三种流行的时间序列Transformer模型中，并在三个真实世界基准数据集上进行全面实验，结果表明性能显著提升，凸显了TEAFormers在尖端时间序列预测中的潜力。