Recent works in multimodal recommendations, which leverage diverse modal information to address data sparsity and enhance recommendation accuracy, have garnered considerable interest. Two key processes in multimodal recommendations are modality fusion and representation learning. Previous approaches in modality fusion often employ simplistic attentive or pre-defined strategies at early or late stages, failing to effectively handle irrelevant information among modalities. In representation learning, prior research has constructed heterogeneous and homogeneous graph structures encapsulating user-item, user-user, and item-item relationships to better capture user interests and item profiles. Modality fusion and representation learning were considered as two independent processes in previous work. In this paper, we reveal that these two processes are complementary and can support each other. Specifically, powerful representation learning enhances modality fusion, while effective fusion improves representation quality. Stemming from these two processes, we introduce a COmposite grapH convolutional nEtwork with dual-stage fuSION for the multimodal recommendation, named COHESION. Specifically, it introduces a dual-stage fusion strategy to reduce the impact of irrelevant information, refining all modalities using ID embedding in the early stage and fusing their representations at the late stage. It also proposes a composite graph convolutional network that utilizes user-item, user-user, and item-item graphs to extract heterogeneous and homogeneous latent relationships within users and items. Besides, it introduces a novel adaptive optimization to ensure balanced and reasonable representations across modalities. Extensive experiments on three widely used datasets demonstrate the significant superiority of COHESION over various competitive baselines.

翻译：近年来，多模态推荐通过利用多样化的模态信息来缓解数据稀疏性问题并提升推荐准确性，已引起了广泛关注。多模态推荐中的两个关键过程是模态融合与表示学习。先前在模态融合方面的工作通常采用简单的注意力机制或预定义策略，在早期或晚期阶段进行融合，未能有效处理模态间的无关信息。在表示学习方面，已有研究构建了包含用户-物品、用户-用户及物品-物品关系的异质与同质图结构，以更好地捕捉用户兴趣与物品特征。以往工作将模态融合与表示学习视为两个独立的过程。本文揭示这两个过程是互补的，能够相互促进：强大的表示学习可增强模态融合，而有效的融合则能提升表示质量。基于这两个过程，我们提出了一种用于多模态推荐的双阶段融合复合图卷积网络，命名为COHESION。具体而言，它引入了一种双阶段融合策略以降低无关信息的影响：在早期阶段利用ID嵌入精炼所有模态，并在晚期阶段融合其表示。同时，它提出了一种复合图卷积网络，利用用户-物品、用户-用户及物品-物品图来提取用户与物品内部的异质及同质潜在关系。此外，它还引入了一种新颖的自适应优化方法，以确保跨模态表示的平衡性与合理性。在三个广泛使用的数据集上进行的大量实验表明，COHESION相较于多种竞争基线方法具有显著优越性。