We introduce LatentTimePFN (LaT-PFN), a foundational Time Series model with a strong embedding space that enables zero-shot forecasting. To achieve this, we perform in-context learning in latent space utilizing a novel integration of the Prior-data Fitted Networks (PFN) and Joint Embedding Predictive Architecture (JEPA) frameworks. We leverage the JEPA framework to create a prediction-optimized latent representation of the underlying stochastic process that generates time series and combines it with contextual learning, using a PFN. Furthermore, we improve on preceding works by utilizing related time series as a context and introducing a normalized abstract time axis. This reduces training time and increases the versatility of the model by allowing any time granularity and forecast horizon. We show that this results in superior zero-shot predictions compared to established baselines. We also demonstrate our latent space produces informative embeddings of both individual time steps and fixed-length summaries of entire series. Finally, we observe the emergence of multi-step patch embeddings without explicit training, suggesting the model actively learns discrete tokens that encode local structures in the data, analogous to vision transformers.

翻译：本文介绍了LatentTimePFN（LaT-PFN），这是一种具有强大嵌入空间的基础时间序列模型，能够实现零样本预测。为实现这一目标，我们通过新颖地整合先验数据拟合网络（PFN）与联合嵌入预测架构（JEPA）框架，在潜在空间中进行上下文学习。我们利用JEPA框架创建对生成时间序列的底层随机过程的预测优化潜在表示，并结合上下文学习（使用PFN）。此外，我们通过利用相关时间序列作为上下文并引入归一化抽象时间轴，改进了先前的工作。这减少了训练时间，并通过允许任意时间粒度和预测范围提高了模型的通用性。我们证明，与现有基线相比，该方法能产生更优的零样本预测结果。我们还展示了我们的潜在空间能够为单个时间步长和整个序列的固定长度摘要生成信息丰富的嵌入。最后，我们观察到在没有显式训练的情况下出现了多步补丁嵌入，这表明模型主动学习了编码数据中局部结构的离散标记，类似于视觉Transformer。