In recent years, AI-Generated Content (AIGC) has witnessed rapid advancements, facilitating the generation of music, images, and other forms of artistic expression across various industries. However, researches on general multi-modal music generation model remain scarce. To fill this gap, we propose a multi-modal music generation framework Mozart's Touch. It could generate aligned music with the cross-modality inputs, such as images, videos and text. Mozart's Touch is composed of three main components: Multi-modal Captioning Module, Large Language Model (LLM) Understanding & Bridging Module, and Music Generation Module. Unlike traditional approaches, Mozart's Touch requires no training or fine-tuning pre-trained models, offering efficiency and transparency through clear, interpretable prompts. We also introduce "LLM-Bridge" method to resolve the heterogeneous representation problems between descriptive texts of different modalities. We conduct a series of objective and subjective evaluations on the proposed model, and results indicate that our model surpasses the performance of current state-of-the-art models. Our codes and examples is availble at: https://github.com/WangTooNaive/MozartsTouch

翻译：近年来，人工智能生成内容（AIGC）在各行业中推动音乐、图像及其他艺术表达形式的生成方面取得了快速进展。然而，针对通用多模态音乐生成模型的研究仍然匮乏。为弥补这一空白，我们提出了一种多模态音乐生成框架Mozart's Touch。该框架能够根据跨模态输入（如图像、视频和文本）生成与之对齐的音乐。Mozart's Touch由三个主要组件构成：多模态字幕模块、大型语言模型（LLM）理解与桥接模块，以及音乐生成模块。与传统方法不同，Mozart's Touch无需对预训练模型进行训练或微调，通过清晰可解释的提示词实现高效性与透明性。我们还引入了称为“LLM-Bridge”的方法，用于解决不同模态描述性文本之间的异构表示问题。我们对所提模型进行了一系列客观与主观评估，结果表明其性能超越了当前最优模型。代码及示例可访问：https://github.com/WangTooNaive/MozartsTouch