To leverage the full potential of multimodal data, we need representations that go beyond the state-of-the-art alignment and fusion approaches and exploit all cross-modal interactions without sacrificing modality-specific information. Learning disentangled representations is a principled way to identify these underlying shared and unique factors that are hidden in observational data. However, while multimodal disentanglement is a compelling paradigm, existing methods are largely confined to the two-modality regime due to its inherent scalability bottleneck. To address this, we propose RePercENT, a self-supervised framework designed to surpass these limitations and unlocks scalable pairwise disentanglement beyond two modalities. Through a multimodal `plug-and-play' architecture, our approach operates directly on pre-extracted embeddings, eliminating the need for extensive joint pre-training while making no assumptions regarding the underlying modalities or foundation model backbones. Moreover, we introduce a joint optimization objective for simultaneously deriving the shared and unique components, and provide formal theoretical guarantees that characterize the optimality of our solution. Across diverse modalities and tasks, RePercENT successfully recovers disentangled components while maintaining competitive performance and significantly reducing computational complexity.

翻译：为充分发挥多模态数据的潜力，我们需要超越现有对齐与融合方法的表现，在保留模态特定信息的同时，充分利用所有跨模态交互。学习解耦表示是识别观测数据中隐藏的共享与特有因素的原理性方法。然而，尽管多模态解耦是一个极具吸引力的范式，现有方法由于其固有的可扩展性瓶颈，大多局限于双模态场景。为此，我们提出RePercENT——一个旨在突破这些限制的自监督框架，解锁超越双模态的可扩展成对解耦能力。通过多模态“即插即用”架构，我们的方法直接作用于预提取的嵌入，既无需大量联合预训练，也不对底层模态或基础模型骨架做任何假设。此外，我们引入了一个联合优化目标，可同时推导共享与特有成分，并提供了形式化的理论保证来刻画我们解的最优性。在多种模态与任务中，RePercENT成功恢复了各解耦成分，同时保持了有竞争力的性能，并显著降低了计算复杂度。