In this paper, we introduce PESTO, a self-supervised learning approach for single-pitch estimation using a Siamese architecture. Our model processes individual frames of a Variable-$Q$ Transform (VQT) and predicts pitch distributions. The neural network is designed to be equivariant to translations, notably thanks to a Toeplitz fully-connected layer. In addition, we construct pitch-shifted pairs by translating and cropping the VQT frames and train our model with a novel class-based transposition-equivariant objective, eliminating the need for annotated data. Thanks to this architecture and training objective, our model achieves remarkable performances while being very lightweight ($130$k parameters). Evaluations on music and speech datasets (MIR-1K, MDB-stem-synth, and PTDB) demonstrate that PESTO not only outperforms self-supervised baselines but also competes with supervised methods, exhibiting superior cross-dataset generalization. Finally, we enhance PESTO's practical utility by developing a streamable VQT implementation using cached convolutions. Combined with our model's low latency (less than 10 ms) and minimal parameter count, this makes PESTO particularly suitable for real-time applications.

翻译：本文提出PESTO，一种基于孪生网络架构的自监督单音高估计方法。我们的模型处理可变Q变换（VQT）的独立帧并预测音高分布。该神经网络通过托普利茨全连接层实现了对平移的等变性。此外，我们通过对VQT帧进行平移和裁剪构建音高偏移样本对，并采用新颖的基于类别的转置等变目标训练模型，从而无需标注数据。得益于该架构与训练目标，我们的模型在保持极轻量化（13万参数）的同时取得了卓越性能。在音乐与语音数据集（MIR-1K、MDB-stem-synth和PTDB）上的评估表明，PESTO不仅优于自监督基线方法，还能与监督方法竞争，并展现出更优异的跨数据集泛化能力。最后，我们通过开发基于缓存卷积的可流式化VQT实现，增强了PESTO的实际应用价值。结合模型低于10毫秒的低延迟与极少的参数量，这些特性使得PESTO特别适用于实时应用场景。