Time series modeling is a well-established problem, which often requires that methods (1) expressively represent complicated dependencies, (2) forecast long horizons, and (3) efficiently train over long sequences. State-space models (SSMs) are classical models for time series, and prior works combine SSMs with deep learning layers for efficient sequence modeling. However, we find fundamental limitations with these prior approaches, proving their SSM representations cannot express autoregressive time series processes. We thus introduce SpaceTime, a new state-space time series architecture that improves all three criteria. For expressivity, we propose a new SSM parameterization based on the companion matrix -- a canonical representation for discrete-time processes -- which enables SpaceTime's SSM layers to learn desirable autoregressive processes. For long horizon forecasting, we introduce a "closed-loop" variation of the companion SSM, which enables SpaceTime to predict many future time-steps by generating its own layer-wise inputs. For efficient training and inference, we introduce an algorithm that reduces the memory and compute of a forward pass with the companion matrix. With sequence length $\ell$ and state-space size $d$, we go from $\tilde{O}(d \ell)$ na\"ively to $\tilde{O}(d + \ell)$. In experiments, our contributions lead to state-of-the-art results on extensive and diverse benchmarks, with best or second-best AUROC on 6 / 7 ECG and speech time series classification, and best MSE on 14 / 16 Informer forecasting tasks. Furthermore, we find SpaceTime (1) fits AR($p$) processes that prior deep SSMs fail on, (2) forecasts notably more accurately on longer horizons than prior state-of-the-art, and (3) speeds up training on real-world ETTh1 data by 73% and 80% relative wall-clock time over Transformers and LSTMs.

翻译：时间序列建模是一个成熟的问题，通常要求方法（1）表达复杂依赖关系，（2）预测长周期未来，（3）在长序列上高效训练。状态空间模型（SSM）是经典的时间序列模型，先前的工作将SSM与深度学习层结合用于高效序列建模。然而，我们发现这些先前方法存在根本性局限，证明其SSM表示无法表达自回归时间序列过程。因此，我们引入SpaceTime——一种改进所有三个标准的新型状态空间时间序列架构。在表达性方面，我们提出基于伴随矩阵（离散时间过程的规范表示）的新SSM参数化，使SpaceTime的SSM层能够学习理想的自回归过程。对于长周期预测，我们引入伴随SSM的"闭环"变体，使SpaceTime能通过生成自身的层内输入来预测多个未来时间步。为高效训练和推理，我们提出一种算法，减少伴随矩阵前向传播的内存和计算量。当序列长度为$\ell$、状态空间大小为$d$时，我们从朴素复杂度$\tilde{O}(d \ell)$降至$\tilde{O}(d + \ell)$。实验中，我们的贡献在广泛且多样化的基准上取得最先进结果：在6/7个ECG和语音时间序列分类任务中排名第一或第二的AUROC，以及在14/16个Informer预测任务中取得最佳MSE。此外，我们发现SpaceTime（1）能够拟合先前深度SSM无法处理的AR($p$)过程，（2）在更长周期上预测精度显著优于先前最先进方法，（3）在真实ETTh1数据上训练相对Transformer和LSTM的墙钟时间分别加速73%和80%。