In recent years, research on music transcription has focused mainly on architecture design and instrument-specific data acquisition. With the lack of availability of diverse datasets, progress is often limited to solo-instrument tasks such as piano transcription. Several works have explored multi-instrument transcription as a means to bolster the performance of models on low-resource tasks, but these methods face the same data availability issues. We propose Timbre-Trap, a novel framework which unifies music transcription and audio reconstruction by exploiting the strong separability between pitch and timbre. We train a single U-Net to simultaneously estimate pitch salience and reconstruct complex spectral coefficients, selecting between either output during the decoding stage via a simple switch mechanism. In this way, the model learns to produce coefficients corresponding to timbre-less audio, which can be interpreted as pitch salience. We demonstrate that the framework leads to performance comparable to state-of-the-art instrument-agnostic transcription methods, while only requiring a small amount of annotated data.

翻译：近年来，音乐转录研究主要聚焦于架构设计与特定乐器的数据采集。由于缺乏多样化数据集，研究进展常局限于钢琴转录等单一乐器任务。部分工作探索了多乐器转录以增强低资源任务模型性能，但这类方法同样面临数据可用性问题。我们提出新型框架"Timbre-Trap"，通过利用音高与音色间的强可分离性，将音乐转录与音频重建统一于一体。该框架训练单一U-Net网络同步实现音高显著性估计与复频谱系数重建，并在解码阶段通过简单开关机制在两种输出间切换。通过这种方式，模型学会生成对应无音色音频的系数（可解释为音高显著性）。实验证明，该框架在仅需少量标注数据的情况下，性能即可媲美当前最先进的乐器无关转录方法。