Not all positive pairs are beneficial to time series contrastive learning. In this paper, we study two types of bad positive pairs that impair the quality of time series representation learned through contrastive learning ($i.e.$, noisy positive pair and faulty positive pair). We show that, with the presence of noisy positive pairs, the model tends to simply learn the pattern of noise (Noisy Alignment). Meanwhile, when faulty positive pairs arise, the model spends considerable efforts aligning non-representative patterns (Faulty Alignment). To address this problem, we propose a Dynamic Bad Pair Mining (DBPM) algorithm, which reliably identifies and suppresses bad positive pairs in time series contrastive learning. DBPM utilizes a memory module to track the training behavior of each positive pair along training process. This allows us to identify potential bad positive pairs at each epoch based on their historical training behaviors. The identified bad pairs are then down-weighted using a transformation module. Our experimental results show that DBPM effectively mitigates the negative impacts of bad pairs, and can be easily used as a plug-in to boost performance of state-of-the-art methods. Codes will be made publicly available.

翻译：并非所有正样本对都对时间序列对比学习有益。本文研究了两类损害对比学习时间序列表示质量的“不良正样本对”（即噪声正样本对有缺陷正样本对）。我们证明，当存在噪声正样本对时，模型倾向于简单学习噪声模式（噪声对齐）；而当出现有缺陷正样本对时，模型会耗费大量精力对齐非代表性模式（有缺陷对齐）。为解决这一问题，我们提出了一种动态不良正样本对挖掘（DBPM）算法，该算法能够可靠地识别并抑制时间序列对比学习中的不良正样本对。DBPM利用记忆模块在训练过程中追踪每个正样本对的训练行为，从而基于其历史训练行为在每个训练周期识别潜在的不良正样本对。随后通过变换模块对识别出的不良正样本对进行降权处理。实验结果表明，DBPM能有效缓解不良正样本对的负面影响，并可便捷地作为即插即用模块提升现有最优方法的性能。相关代码将公开发布。