This study investigates self-supervised learning techniques to obtain representations of Event Sequences. It is a key modality in various applications, including but not limited to banking, e-commerce, and healthcare. We perform a comprehensive study of generative and contrastive approaches in self-supervised learning, applying them both independently. We find that there is no single supreme method. Consequently, we explore the potential benefits of combining these approaches. To achieve this goal, we introduce a novel method that aligns generative and contrastive embeddings as distinct modalities, drawing inspiration from contemporary multimodal research. Generative and contrastive approaches are often treated as mutually exclusive, leaving a gap for their combined exploration. Our results demonstrate that this aligned model performs at least on par with, and mostly surpasses, existing methods and is more universal across a variety of tasks. Furthermore, we demonstrate that self-supervised methods consistently outperform the supervised approach on our datasets.

翻译：本研究探讨了自监督学习技术在获取事件序列表示方面的应用。事件序列是银行、电子商务和医疗保健等多种应用中的一个关键模态。我们对自监督学习中的生成式和对比式方法进行了全面研究，并独立应用了这两种方法。研究发现，不存在单一的最优方法。因此，我们探索了结合这些方法的潜在优势。为实现这一目标，我们借鉴当代多模态研究的思路，提出了一种新方法，将生成式和对比式嵌入作为不同模态进行对齐。生成式和对比式方法通常被视为互斥的，这导致了对它们联合探索的空白。我们的结果表明，这种对齐模型至少在性能上与现有方法相当，且大多优于现有方法，并且在各种任务中更具通用性。此外，我们证明，在我们的数据集上，自监督方法始终优于监督方法。