Recent works in multimodal recommendations, which leverage diverse modal information to address data sparsity and enhance recommendation accuracy, have garnered considerable interest. Two key processes in multimodal recommendations are modality fusion and representation learning. Previous approaches in modality fusion often employ simplistic attentive or pre-defined strategies at early or late stages, failing to effectively handle irrelevant information among modalities. In representation learning, prior research has constructed heterogeneous and homogeneous graph structures encapsulating user-item, user-user, and item-item relationships to better capture user interests and item profiles. Modality fusion and representation learning were considered as two independent processes in previous work. In this paper, we reveal that these two processes are complementary and can support each other. Specifically, powerful representation learning enhances modality fusion, while effective fusion improves representation quality. Stemming from these two processes, we introduce a COmposite grapH convolutional nEtwork with dual-stage fuSION for the multimodal recommendation, named COHESION. Specifically, it introduces a dual-stage fusion strategy to reduce the impact of irrelevant information, refining all modalities using ID embedding in the early stage and fusing their representations at the late stage. It also proposes a composite graph convolutional network that utilizes user-item, user-user, and item-item graphs to extract heterogeneous and homogeneous latent relationships within users and items. Besides, it introduces a novel adaptive optimization to ensure balanced and reasonable representations across modalities. Extensive experiments on three widely used datasets demonstrate the significant superiority of COHESION over various competitive baselines.

翻译：近年来，多模态推荐系统通过利用多样化的模态信息来缓解数据稀疏性问题并提升推荐准确性，已引起广泛关注。多模态推荐中的两个关键过程是模态融合与表示学习。先前研究在模态融合方面通常采用早期或晚期阶段的简单注意力机制或预定义策略，未能有效处理模态间的无关信息。在表示学习方面，已有研究构建了包含用户-物品、用户-用户及物品-物品关系的异质性与同质性图结构，以更好地捕捉用户兴趣与物品特征。过往工作将模态融合与表示学习视为两个独立过程。本文揭示这两个过程具有互补性且能相互促进：强大的表示学习可增强模态融合效果，而有效的融合又能提升表示质量。基于此，我们提出一种面向多模态推荐的双阶段融合复合图卷积网络——COHESION。具体而言，该模型引入双阶段融合策略以降低无关信息影响：早期阶段利用ID嵌入精炼所有模态，晚期阶段融合其表示；同时设计复合图卷积网络，通过用户-物品、用户-用户及物品-物品图提取用户与物品间的异质性与同质性潜在关系。此外，模型提出新型自适应优化机制，确保跨模态表示的平衡性与合理性。在三个广泛使用的数据集上的大量实验表明，COHESION相较于多种竞争基线方法具有显著优越性。