Many deep learning models have achieved dominant performance on the offline beat tracking task. However, online beat tracking, in which only the past and present input features are available, still remains challenging. In this paper, we propose BEAt tracking Streaming Transformer (BEAST), an online joint beat and downbeat tracking system based on the streaming Transformer. To deal with online scenarios, BEAST applies contextual block processing in the Transformer encoder. Moreover, we adopt relative positional encoding in the attention layer of the streaming Transformer encoder to capture relative timing position which is critically important information in music. Carrying out beat and downbeat experiments on benchmark datasets for a low latency scenario with maximum latency under 50 ms, BEAST achieves an F1-measure of 80.04% in beat and 52.73% in downbeat, which is a substantial improvement of about 5 and 13 percentage points over the state-of-the-art online beat and downbeat tracking model.
翻译:许多深度学习模型在离线节拍追踪任务中已展现出卓越性能。然而,仅依赖过去与当前输入特征的在线节拍追踪仍具挑战性。本文提出节拍追踪流式Transformer(BEAST)——一种基于流式Transformer的在线节拍与强拍联合追踪系统。为应对在线场景,BEAST在Transformer编码器中采用上下文块处理机制。同时,我们在流式Transformer编码器的注意力层引入相对位置编码,以捕获音乐中至关重要的相对时序位置信息。在最大延迟低于50毫秒的低延迟场景下,基于基准数据集的节拍与强拍实验表明,BEAST的节拍F1值达80.04%,强拍F1值达52.73%,较当前最优在线节拍与强拍追踪模型分别提升约5个及13个百分点。