This paper introduces the MPS (Model Prediction Set), a novel framework for online model selection for nonstationary time series. Classical model selection methods, such as information criteria and cross-validation, rely heavily on the stationarity assumption and often fail in dynamic environments which undergo gradual or abrupt changes over time. Yet real-world data are rarely stationary, and model selection under nonstationarity remains a largely open problem. To tackle this challenge, we combine conformal inference with model confidence sets to develop a procedure that adaptively selects models best suited to the evolving dynamics at any given time. Concretely, the MPS updates in real time a confidence set of candidate models that covers the best model for the next time period with a specified long-run probability, while adapting to nonstationarity of unknown forms. Through simulations and real-world data analysis, we demonstrate that MPS reliably and efficiently identifies optimal models under nonstationarity, an essential capability lacking in offline methods. Moreover, MPS frequently produces high-quality sets with small cardinality, whose evolution offers deeper insights into changing dynamics. As a generic framework, MPS accommodates any data-generating process, data structure, model class, training method, and evaluation metric, making it broadly applicable across diverse problem settings.

翻译：本文提出了一种用于非平稳时间序列在线模型选择的新框架——MPS（模型预测集）。经典的模型选择方法，如信息准则和交叉验证，严重依赖于平稳性假设，在随时间经历渐变或突变动态环境中常常失效。然而现实世界的数据很少是平稳的，非平稳性下的模型选择在很大程度上仍是一个悬而未决的问题。为应对这一挑战，我们将共形推断与模型置信集相结合，开发出一种能够自适应地选择在任意给定时刻最适应演化动态的模型的方法。具体而言，MPS实时更新候选模型的置信集，该置信集以指定的长期概率覆盖下一时间段的最佳模型，同时适应未知形式的非平稳性。通过仿真和真实世界数据分析，我们证明MPS能够可靠且高效地识别非平稳性下的最优模型，这是离线方法所缺乏的关键能力。此外，MPS通常能生成基数较小的高质量集合，其演化过程为理解动态变化提供了更深入的洞见。作为一个通用框架，MPS适用于任何数据生成过程、数据结构、模型类别、训练方法和评估指标，使其能够广泛应用于多样化的问题场景。