Transformer architectures have been widely adopted for time series forecasting, yet whether the representational mechanisms that make them powerful in NLP actually engage on time series data remains unexplored. The persistent competitiveness of simple linear models such as DLinear has fueled ongoing debate, but no mechanistic explanation for this phenomenon has been offered. We address this gap by applying sparse autoencoders (SAEs), a tool from mechanistic interpretability, to probe the internal representations of PatchTST. We first establish that a single-layer, narrow-dimensional transformer matches the forecasting performance of deeper configurations across commonly used benchmarks. We then train SAEs on the post-GELU intermediate FFN activations with dictionary sizes ranging from 0.5x to 4.0x the native dimensionality. Expanding the dictionary yields negligible downstream performance change (average 0.214%), with large portions of overcomplete dictionaries remaining inactive. Targeted causal interventions on dominant latent features produce minimal forecast perturbation. Across all evaluated settings, we observe no empirical evidence that the analyzed FFN representations rely on strong superposition. Instead, the representations remain sparse, stable under aggressive dictionary expansion, and largely insensitive to latent interventions. These results demonstrate that superposition is not necessary for competitive performance on standard forecasting benchmarks, suggesting they may not demand the rich compositional representations that drive transformer success in language modeling, and helping explain the persistent competitiveness of simple linear models

翻译：Transformer架构已被广泛应用于时间序列预测，但使其在自然语言处理中表现强大的表示机制是否实际作用于时间序列数据，这一问题尚未得到探究。DLinear等简单线性模型的持续竞争力引发了长期争论，但迄今缺乏对此现象的机制性解释。我们通过应用稀疏自编码器（一种来自机制可解释性的工具）来探测PatchTST的内部表示，从而填补这一空白。我们首先证明，在常用基准测试中，单层窄维度Transformer在预测性能上可与更深层配置相匹敌。随后，我们在后GELU的中间前馈神经网络激活上训练SAE，字典大小从原生维度的0.5倍到4.0倍不等。扩大字典仅带来可忽略的后续性能变化（平均0.214%），且过完备字典中存在大量未激活的部分。对主导潜在特征进行定向因果干预仅产生微小的预测扰动。在所有评估设置中，我们未观察到实证证据表明所分析的前馈神经网络表示依赖于强叠加。相反，这些表示保持稀疏性、在激进的字典扩展下保持稳定，且对潜在干预大体不敏感。这些结果表明，在标准预测基准上实现竞争性能并不需要叠加，这或许意味着这些任务可能不需要驱动Transformer在语言建模中取得成功的丰富组合表示，并有助于解释简单线性模型的持续竞争力。