Recent contrastive methods show significant improvement in self-supervised learning in several domains. In particular, contrastive methods are most effective where data augmentation can be easily constructed e.g. in computer vision. However, they are less successful in domains without established data transformations such as time series data. In this paper, we propose a novel self-supervised learning framework that combines contrastive learning with neural processes. It relies on recent advances in neural processes to perform time series forecasting. This allows to generate augmented versions of data by employing a set of various sampling functions and, hence, avoid manually designed augmentations. We extend conventional neural processes and propose a new contrastive loss to learn times series representations in a self-supervised setup. Therefore, unlike previous self-supervised methods, our augmentation pipeline is task-agnostic, enabling our method to perform well across various applications. In particular, a ResNet with a linear classifier trained using our approach is able to outperform state-of-the-art techniques across industrial, medical and audio datasets improving accuracy over 10% in ECG periodic data. We further demonstrate that our self-supervised representations are more efficient in the latent space, improving multiple clustering indexes and that fine-tuning our method on 10% of labels achieves results competitive to fully-supervised learning.

翻译：最近的对比方法显示,在多个领域自我监督的学习有显著改进。 特别是, 对比方法最为有效, 数据增强可以很容易地构建, 比如计算机视觉。 但是, 在没有固定的数据转换( 如时间序列数据) 的领域, 对比方法不太成功 。 在本文中, 我们提出一个新的自我监督学习框架, 将对比学习与神经过程相结合。 它依靠神经过程的最新进步来进行时间序列预测。 这样可以使用一套各种取样功能来生成扩大的数据版本, 从而避免人工设计的增强。 我们扩展常规神经过程, 并提议新的对比损失, 以在自我监督的设置中学习时间序列显示。 因此, 与先前的自我监督的方法不同, 我们的增强管道是任务- 敏感化的, 使得我们的方法能够在各种应用中很好地发挥作用。 特别是, 使用我们的方法经过培训的线性分类器的ResNet 能够超越各种工业、 医学和音频数据集的状态技术, 从而可以提高ECG 定期数据的准确度。 我们进一步展示了常规神经过程, 并提议新的时间序列显示, 我们的自我监督方法在10 的自我监督模型中, 我们的自我更新了10 的自我监督模型中, 我们的自我监督模型的模型中, 学习了10 。