Despite the success of Transformer-based models in the time-series prediction (TSP) tasks, the existing Transformer architecture still face limitations and the literature lacks comprehensive explorations into alternative architectures. To address these challenges, we propose AutoFormer-TS, a novel framework that leverages a comprehensive search space for Transformer architectures tailored to TSP tasks. Our framework introduces a differentiable neural architecture search (DNAS) method, AB-DARTS, which improves upon existing DNAS approaches by enhancing the identification of optimal operations within the architecture. AutoFormer-TS systematically explores alternative attention mechanisms, activation functions, and encoding operations, moving beyond the traditional Transformer design. Extensive experiments demonstrate that AutoFormer-TS consistently outperforms state-of-the-art baselines across various TSP benchmarks, achieving superior forecasting accuracy while maintaining reasonable training efficiency.

翻译：尽管基于Transformer的模型在时间序列预测任务中取得了成功，但现有Transformer架构仍存在局限性，且文献缺乏对替代架构的全面探索。为应对这些挑战，我们提出了AutoFormer-TS——一个利用专为时间序列预测任务定制的Transformer架构综合搜索空间的新型框架。该框架引入了可微分神经架构搜索方法AB-DARTS，通过增强对架构内最优操作的识别能力，改进了现有可微分神经架构搜索方法。AutoFormer-TS系统性地探索了替代注意力机制、激活函数与编码操作，突破了传统Transformer的设计范式。大量实验表明，AutoFormer-TS在多个时间序列预测基准测试中持续优于现有先进基线模型，在保持合理训练效率的同时实现了更优的预测精度。