We present JASCO, a temporally controlled text-to-music generation model utilizing both symbolic and audio-based conditions. JASCO can generate high-quality music samples conditioned on global text descriptions along with fine-grained local controls. JASCO is based on the Flow Matching modeling paradigm together with a novel conditioning method. This allows music generation controlled both locally (e.g., chords) and globally (text description). Specifically, we apply information bottleneck layers in conjunction with temporal blurring to extract relevant information with respect to specific controls. This allows the incorporation of both symbolic and audio-based conditions in the same text-to-music model. We experiment with various symbolic control signals (e.g., chords, melody), as well as with audio representations (e.g., separated drum tracks, full-mix). We evaluate JASCO considering both generation quality and condition adherence, using both objective metrics and human studies. Results suggest that JASCO is comparable to the evaluated baselines considering generation quality while allowing significantly better and more versatile controls over the generated music. Samples are available on our demo page https://pages.cs.huji.ac.il/adiyoss-lab/JASCO.

翻译：我们提出了JASCO，一种利用符号和音频条件的时序可控文本到音乐生成模型。JASCO能够基于全局文本描述以及细粒度局部控制生成高质量音乐样本。该模型基于流匹配建模范式并结合了一种新颖的条件控制方法，从而实现了局部（如和弦）与全局（文本描述）相结合的音乐生成控制。具体而言，我们通过信息瓶颈层结合时序模糊化技术，针对特定控制信号提取相关信息。这使得同一文本到音乐模型能够同时融合符号与音频条件。我们实验了多种符号控制信号（如和弦、旋律）以及音频表示（如分离的鼓声轨、完整混音）。我们通过客观指标和人工评估对JASCO的生成质量与条件遵循度进行了综合评估。结果表明，在保持与基线模型相当的生成质量的同时，JASCO能够对生成音乐实现显著更优且更多样化的控制。生成样本可在演示页面 https://pages.cs.huji.ac.il/adiyoss-lab/JASCO 获取。