In this paper, we propose Emotionally paired Music and Image Dataset (EMID), a novel dataset designed for the emotional matching of music and images, to facilitate auditory-visual cross-modal tasks such as generation and retrieval. Unlike existing approaches that primarily focus on semantic correlations or roughly divided emotional relations, EMID emphasizes the significance of emotional consistency between music and images using an advanced 13-dimension emotional model. By incorporating emotional alignment into the dataset, it aims to establish pairs that closely align with human perceptual understanding, thereby raising the performance of auditory-visual cross-modal tasks. We also design a supplemental module named EMI-Adapter to optimize existing cross-modal alignment methods. To validate the effectiveness of the EMID, we conduct a psychological experiment, which has demonstrated that considering the emotional relationship between the two modalities effectively improves the accuracy of matching in abstract perspective. This research lays the foundation for future cross-modal research in domains such as psychotherapy and contributes to advancing the understanding and utilization of emotions in cross-modal alignment. The EMID dataset is available at https://github.com/ecnu-aigc/EMID.

翻译：本文提出情感配对音乐与图像数据集（EMID），这是一个专为音乐与图像情感匹配设计的新颖数据集，旨在促进诸如生成与检索等视听跨模态任务。与现有主要关注语义相关性或粗略划分情感关系的方法不同，EMID采用先进的13维情感模型，强调音乐与图像之间情感一致性的重要性。通过将情感对齐纳入数据集，其目标是建立与人类感知理解紧密对齐的配对，从而提升视听跨模态任务的性能。我们还设计了一个名为EMI-Adapter的补充模块，以优化现有跨模态对齐方法。为验证EMID的有效性，我们进行了一项心理实验，结果表明，在抽象视角下，考虑两种模态之间的情感关系能有效提升匹配准确性。本研究为未来诸如心理治疗等领域的跨模态研究奠定了基础，并有助于推进对跨模态对齐中情感的理解与利用。EMID数据集可在https://github.com/ecnu-aigc/EMID获取。