When used with deep learning, the symbolic music modality is often coupled with language model architectures. To do so, the music needs to be tokenized, i.e. converted into a sequence of discrete tokens. This can be achieved by different approaches, as music can be composed of simultaneous tracks, of simultaneous notes with several attributes. Until now, the proposed tokenizations rely on small vocabularies of tokens describing the note attributes and time events, resulting in fairly long token sequences, and a sub-optimal use of the embedding space of language models. Recent research has put efforts on reducing the overall sequence length by merging embeddings or combining tokens. In this paper, we show that Byte Pair Encoding, a compression technique widely used for natural language, significantly decreases the sequence length while increasing the vocabulary size. By doing so, we leverage the embedding capabilities of such models with more expressive tokens, resulting in both better results and faster inference in generation and classification tasks. The source code is shared on Github, along with a companion website. Finally, BPE is directly implemented in MidiTok, allowing the reader to easily benefit from this method.

翻译：在深度学习应用中，符号音乐模态常与语言模型架构结合使用。为此，需要将音乐进行分词处理，即转化为离散词元序列。由于音乐可由多轨并行演奏及多属性同时发音构成，因此可采用多种分词策略。目前提出的分词方案依赖于描述音符属性和时间事件的较小词表，导致生成的词元序列较长，且未能充分利用语言模型的嵌入空间。近期研究致力于通过合并嵌入或组合词元来缩短整体序列长度。本文证明，自然语言处理中广泛使用的压缩技术——字节对编码（Byte Pair Encoding），能在显著增大词表容量的同时有效缩短序列长度。该方法通过更具表现力的词元充分挖掘模型嵌入能力，在生成与分类任务中同时提升效果表现与推理速度。相关源代码已共享至Github平台，并配有配套网站。此外，BPE技术已直接集成于MidiTok工具包中，便于读者直接应用该方法。