Lyrics alignment gained considerable attention in recent years. State-of-the-art systems either re-use established speech recognition toolkits, or design end-to-end solutions involving a Connectionist Temporal Classification (CTC) loss. However, both approaches suffer from specific weaknesses: toolkits are known for their complexity, and CTC systems use a loss designed for transcription which can limit alignment accuracy. In this paper, we use instead a contrastive learning procedure that derives cross-modal embeddings linking the audio and text domains. This way, we obtain a novel system that is simple to train end-to-end, can make use of weakly annotated training data, jointly learns a powerful text model, and is tailored to alignment. The system is not only the first to yield an average absolute error below 0.2 seconds on the standard Jamendo dataset but it is also robust to other languages, even when trained on English data only. Finally, we release word-level alignments for the JamendoLyrics Multi-Lang dataset.

翻译：歌词对齐近年来受到了广泛关注。现有最先进的系统要么复用成熟的语音识别工具包，要么设计基于连接主义时序分类（CTC）损失的端到端解决方案。然而，这两种方法都存在特定缺陷：工具包以复杂性著称，而CTC系统使用的损失函数专为转录设计，可能限制对齐精度。本文转而采用对比学习流程，该流程能够推导连接音频与文本领域的跨模态嵌入。由此，我们获得了一个新颖的系统，它易于端到端训练、可利用弱标注训练数据、联合学习强大的文本模型，并且专为对齐任务定制。该系统不仅首次在标准Jamendo数据集上实现了低于0.2秒的平均绝对误差，而且即使仅使用英语数据训练，也能对其他语言保持鲁棒性。最后，我们发布了JamendoLyrics多语言数据集的词级对齐结果。