Multimodal Sentiment Analysis (MSA) requires integrating language, acoustic, and visual signals without sacrificing modality-specific sentiment evidence. Existing methods mainly improve either shared-private decomposition or cross-modal interaction. Although effective, both ultimately depend on how shared and modality-specific evidence is organized before prediction. We observe that, under standard shared-private pipelines, modality heterogeneity often induces a branch-imbalance process: dominant shared patterns accumulate in the shared branch, yielding redundant and modality-biased evidence, while repeated interaction and rigid alignment gradually leak shared information into modality-specific channels and weaken discriminative private representations. As a result, the complementarity between shared and private representations is reduced, limiting robust sentiment reasoning. To address this issue, we propose the Dual-Branch Rebalancing Framework (DBR) on top of a standard multimodal decoupling stage. In the shared branch, a Temporal-Structural Factorization (TSF) module disentangles temporal evolution from structural dependencies and adaptively integrates them to reduce shared redundancy. In the private branch, an Anchor-Guided Private Routing (AGPR) module preserves discriminative modality-specific patterns while allowing controlled cross-modal borrowing. A Bidirectional Rebalancing Fusion (BRF) module then reunifies the two regularized branches in a context-aware manner for final prediction. Extensive experiments on CMU-MOSI, CMU-MOSEI, and MIntRec demonstrate that DBR consistently outperforms the compared baselines. Further analyses show that these improvements come from coordinated mitigation of branch imbalance.
翻译:多模态情感分析(MSA)需要整合语言、声学及视觉信号,同时不损失各模态特有的情感证据。现有方法主要改进共享-私有分解或跨模态交互。尽管有效,这两类方法的最终性能都取决于共享证据与模态特定证据在预测前的组织方式。我们观察到,在标准共享-私有流程下,模态异质性常诱发分支失衡过程:主导性共享模式在共享分支中积累,产生冗余且模态偏置的证据;而重复的交互与刚性对齐会逐渐将共享信息泄漏到模态特定通道,削弱判别性私有表征。这导致共享表征与私有表征之间的互补性降低,限制鲁棒的情感推理。为解决该问题,我们在标准多模态解耦阶段之上提出双分支重平衡框架(DBR)。在共享分支中,时序-结构分解(TSF)模块将时间演化与结构依赖性解耦,并自适应整合两者以降低共享冗余。在私有分支中,锚点引导的私有路由(AGPR)模块保留判别性模态特定模式,同时允许受控的跨模态借用。双向重平衡融合(BRF)模块随后以上下文感知方式重新统一两个正则化分支进行最终预测。在CMU-MOSI、CMU-MOSEI及MIntRec上的大量实验表明,DBR持续优于对比基线。进一步分析显示,这些改进源于对分支失衡的协同缓解。