We introduce LatentTimePFN (LaT-PFN), a foundational Time Series model with a strong embedding space that enables zero-shot forecasting. To achieve this, we perform in-context learning in latent space utilizing a novel integration of the Prior-data Fitted Networks (PFN) and Joint Embedding Predictive Architecture (JEPA) frameworks. We leverage the JEPA framework to create a prediction-optimized latent representation of the underlying stochastic process that generates time series and combines it with contextual learning, using a PFN. Furthermore, we improve on preceding works by utilizing related time series as a context and introducing an abstract time axis. This drastically reduces training time and increases the versatility of the model by allowing any time granularity and forecast horizon. We show that this results in superior zero-shot predictions compared to established baselines. We also demonstrate our latent space produces informative embeddings of both individual time steps and fixed-length summaries of entire series. Finally, we observe the emergence of multi-step patch embeddings without explicit training, suggesting the model actively learns discrete tokens that encode local structures in the data, analogous to vision transformers.

翻译：摘要：我们提出LatentTimePFN（LaT-PFN）——一种具有强嵌入空间的基础时间序列模型，能够实现零样本预测。为此，我们在隐空间中利用先验数据拟合网络（PFN）与联合嵌入预测架构（JEPA）框架的新型集成方式执行上下文学习。通过JEPA框架，我们为生成时间序列的潜在随机过程创建了预测优化的隐式表示，并将其与基于PFN的上下文学习相结合。此外，我们通过引入相关时间序列作为上下文及抽象时间轴改进了先前工作。这大幅缩短了训练时间，并通过允许任意时间粒度和预测范围提升了模型的通用性。实验表明，与现有基线相比，该方法在零样本预测中取得了更优性能。我们还证明了隐空间能生成单个时间步骤和整序列固定长度摘要的信息性嵌入。最后，我们观察到多步分块嵌入在未显式训练的情况下涌现，表明模型主动学习编码数据局部结构的离散令牌，这与视觉Transformer具有相似性。