Multimodal learning integrates data from diverse sensors to effectively harness information from different modalities. However, recent studies reveal that joint learning often overfits certain modalities while neglecting others, leading to performance inferior to that of unimodal learning. Although previous efforts have sought to balance modal contributions or combine joint and unimodal learning, thereby mitigating the degradation of weaker modalities with promising outcomes, few have examined the relationship between joint and unimodal learning from an information-theoretic perspective. In this paper, we theoretically analyze modality competition and propose a method for multimodal classification by maximizing the total correlation between multimodal features and labels. By maximizing this objective, our approach alleviates modality competition while capturing inter-modal interactions via feature alignment. Building on Mutual Information Neural Estimation (MINE), we introduce Total Correlation Neural Estimation (TCNE) to derive a lower bound for total correlation. Subsequently, we present TCMax, a hyperparameter-free loss function that maximizes total correlation through variational bound optimization. Extensive experiments demonstrate that TCMax outperforms state-of-the-art joint and unimodal learning approaches. Our code is available at https://github.com/hubaak/TCMax.

翻译：多模态学习整合来自不同传感器的数据，以有效利用来自不同模态的信息。然而，近期研究表明，联合学习常常会过度拟合某些模态而忽略其他模态，导致其性能甚至不及单模态学习。尽管先前的研究尝试平衡模态贡献或结合联合学习与单模态学习，从而在减轻较弱模态性能退化方面取得了有希望的结果，但很少有研究从信息论的角度审视联合学习与单模态学习之间的关系。本文从理论上分析了模态竞争，并提出一种通过最大化多模态特征与标签之间的总相关来实现多模态分类的方法。通过最大化该目标，我们的方法在通过特征对齐捕获模态间交互的同时，缓解了模态竞争。基于互信息神经估计（MINE），我们引入了总相关神经估计（TCNE）以推导总相关的下界。随后，我们提出了TCMax，这是一种通过变分界优化最大化总相关的无超参数损失函数。大量实验表明，TCMax在性能上超越了最先进的联合学习与单模态学习方法。我们的代码可在 https://github.com/hubaak/TCMax 获取。