Multimodal Sentiment Analysis (MSA) aims to identify speakers' sentiment tendencies in multimodal video content, raising serious concerns about privacy risks associated with multimodal data, such as voiceprints and facial images. Recent distributed collaborative learning has been verified as an effective paradigm for privacy preservation in multimodal tasks. However, they often overlook the privacy distinctions among different modalities, struggling to strike a balance between performance and privacy preservation. Consequently, it poses an intriguing question of maximizing multimodal utilization to improve performance while simultaneously protecting necessary modalities. This paper forms the first attempt at modality-specified (i.e., audio and visual) privacy preservation in MSA tasks. We propose a novel Hybrid Distributed cross-modality cGAN framework (HyDiscGAN), which learns multimodality alignment to generate fake audio and visual features conditioned on shareable de-identified textual data. The objective is to leverage the fake features to approximate real audio and visual content to guarantee privacy preservation while effectively enhancing performance. Extensive experiments show that compared with the state-of-the-art MSA model, HyDiscGAN can achieve superior or competitive performance while preserving privacy.

翻译：多模态情感分析（MSA）旨在识别多模态视频内容中说话者的情感倾向，由此引发了关于语音指纹和人脸图像等多模态数据隐私风险的严峻关切。近期分布式协作学习已被验证为多模态任务中隐私保护的有效范式，然而此类方法常忽视不同模态间的隐私差异性，难以在性能与隐私保护间取得平衡。由此引出最大化多模态利用率以提升性能、同时保护必要模态的深刻问题。本文首次尝试在MSA任务中实现模态特指（即音频与视觉）隐私保护，提出一种新颖的混合分布式跨模态条件生成对抗网络框架（HyDiscGAN），该框架学习多模态对齐，从而基于可共享的去标识化文本数据生成伪音频与视觉特征。其目标在于利用伪特征逼近真实音频与视觉内容，在有效提升性能的同时保障隐私保护。大量实验表明，相较于最先进的MSA模型，HyDiscGAN在保持隐私保护的前提下可实现更优或相当的性能。