Existing approaches for learning representations of time-series keep the temporal arrangement of the time-steps intact with the presumption that the original order is the most optimal for learning. However, non-adjacent sections of real-world time-series may have strong dependencies. Accordingly we raise the question: Is there an alternative arrangement for time-series which could enable more effective representation learning? To address this, we propose a simple plug-and-play mechanism called Segment, Shuffle, and Stitch (S3) designed to improve time-series representation learning of existing models. S3 works by creating non-overlapping segments from the original sequence and shuffling them in a learned manner that is the most optimal for the task at hand. It then re-attaches the shuffled segments back together and performs a learned weighted sum with the original input to capture both the newly shuffled sequence along with the original sequence. S3 is modular and can be stacked to create various degrees of granularity, and can be added to many forms of neural architectures including CNNs or Transformers with negligible computation overhead. Through extensive experiments on several datasets and state-of-the-art baselines, we show that incorporating S3 results in significant improvements for the tasks of time-series classification and forecasting, improving performance on certain datasets by up to 68\%. We also show that S3 makes the learning more stable with a smoother training loss curve and loss landscape compared to the original baseline. The code is available at https://github.com/shivam-grover/S3-TimeSeries .

翻译：现有学习时间序列表示的方法通常保持时间步的时序排列不变，其预设前提是原始顺序对学习最为有利。然而，现实世界时间序列的非相邻片段间可能存在强依赖性。基于此，我们提出一个关键问题：是否存在一种替代性的时间序列排列方式，能够实现更有效的表示学习？为解决该问题，我们提出一种名为"分段、混洗与拼接"（S3）的即插即用式简单机制，旨在提升现有模型的时间序列表示学习能力。S3的工作原理是：将原始序列划分为非重叠片段，通过任务导向的可学习方式对其进行最优混洗；随后将混洗后的片段重新拼接，并与原始输入进行可学习的加权求和，以同时捕获新混洗序列与原始序列的信息。S3采用模块化设计，可通过堆叠实现不同粒度级别的处理，并能以可忽略的计算开销集成到包括CNN或Transformer在内的多种神经网络架构中。通过在多个数据集和前沿基线模型上的大量实验，我们证明引入S3能显著提升时间序列分类与预测任务的性能，在某些数据集上的性能提升最高达68%。研究还表明，相较于原始基线模型，S3能使学习过程更稳定，其训练损失曲线与损失景观都更为平滑。代码已开源：https://github.com/shivam-grover/S3-TimeSeries。