In many real-world scenarios, distribution shifts exist in the streaming data across time steps. Many complex sequential data can be effectively divided into distinct regimes that exhibit persistent dynamics. Discovering the shifted behaviors and the evolving patterns underlying the streaming data are important to understand the dynamic system. Existing methods typically train one robust model to work for the evolving data of distinct distributions or sequentially adapt the model utilizing explicitly given regime boundaries. However, there are two challenges: (1) shifts in data streams could happen drastically and abruptly without precursors. Boundaries of distribution shifts are usually unavailable, and (2) training a shared model for all domains could fail to capture varying patterns. This paper aims to solve the problem of sequential data modeling in the presence of sudden distribution shifts that occur without any precursors. Specifically, we design a Bayesian framework, dubbed as T-SaS, with a discrete distribution-modeling variable to capture abrupt shifts of data. Then, we design a model that enable adaptation with dynamic network selection conditioned on that discrete variable. The proposed method learns specific model parameters for each distribution by learning which neurons should be activated in the full network. A dynamic masking strategy is adopted here to support inter-distribution transfer through the overlapping of a set of sparse networks. Extensive experiments show that our proposed method is superior in both accurately detecting shift boundaries to get segments of varying distributions and effectively adapting to downstream forecast or classification tasks.

翻译：在许多实际场景中，流数据随时间步长存在分布偏移。复杂的序列数据可有效划分为多个具有持久动态特性的不同状态区间。揭示流数据背后的偏移行为与演化模式对于理解动态系统至关重要。现有方法通常训练单一鲁棒模型以处理不同分布下的演化数据，或利用显式给出的状态边界逐步调整模型。然而存在两大挑战：（1）数据流中的偏移可能毫无预兆地急剧发生，分布偏移边界通常不可得；（2）为所有域训练共享模型无法捕捉动态变化模式。本文旨在解决无前兆突变分布偏移下的序列数据建模问题。具体而言，我们设计了一个名为T-SaS的贝叶斯框架，通过离散分布建模变量捕捉数据突变。进而设计基于该离散变量的动态网络选择自适应模型。该方法通过学习全连接网络中应激活的神经元，为每种分布学习特定模型参数。采用动态掩码策略通过稀疏网络的重叠支持跨分布迁移。大量实验表明，本方法在精确检测偏移边界以获取不同分布片段，以及有效适应下游预测或分类任务方面均表现优越。