Biosignals offer valuable insights into the physiological states of the human body. Although biosignal modalities differ in functionality, signal fidelity, sensor comfort, and cost, they are often intercorrelated, reflecting the holistic and interconnected nature of human physiology. This opens up the possibility of performing the same tasks using alternative biosignal modalities, thereby improving the accessibility, usability, and adaptability of health monitoring systems. However, the limited availability of large labeled datasets presents challenges for training models tailored to specific tasks and modalities of interest. Unsupervised cross-modal knowledge transfer offers a promising solution by leveraging knowledge from an existing modality to support model training for a new modality. Existing methods are typically based on knowledge distillation, which requires running a teacher model alongside student model training, resulting in high computational and memory overhead. This challenge is further exacerbated by the recent development of foundation models that demonstrate superior performance and generalization across tasks at the cost of large model sizes. To this end, we explore a new framework for unsupervised cross-modal knowledge transfer of biosignals by training a lightweight bridge network to align the intermediate representations and enable information flow between foundation models and across modalities. Specifically, we introduce an efficient strategy for selecting alignment positions where the bridge should be constructed, along with a flexible prototype network as the bridge architecture. Extensive experiments across multiple biosignal modalities, tasks, and datasets show that BioX-Bridge reduces the number of trainable parameters by 88--99\% while maintaining or even improving transfer performance compared to state-of-the-art methods.
翻译:生物信号为人体生理状态提供了宝贵的洞察。尽管不同生物信号模态在功能、信号保真度、传感器舒适度和成本上存在差异,但它们通常相互关联,反映了人体生理的整体性与互联性。这为使用替代生物信号模态执行相同任务提供了可能,从而提升了健康监测系统的可及性、可用性和适应性。然而,大规模标注数据集的有限可用性为针对特定任务和关注模态训练定制化模型带来了挑战。无监督跨模态知识迁移通过利用现有模态的知识来支持新模态的模型训练,提供了一种有前景的解决方案。现有方法通常基于知识蒸馏,这需要在学生模型训练的同时运行教师模型,导致较高的计算和内存开销。近期基础模型的发展进一步加剧了这一挑战,这些模型以庞大的模型规模为代价,在跨任务中展现出卓越的性能和泛化能力。为此,我们探索了一种新的生物信号无监督跨模态知识迁移框架,通过训练一个轻量级的桥接网络来对齐中间表示,并实现基础模型之间及跨模态的信息流动。具体而言,我们引入了一种高效的策略来选择构建桥接的对齐位置,并采用一个灵活的原型网络作为桥接架构。在多种生物信号模态、任务和数据集上的大量实验表明,与最先进方法相比,BioX-Bridge 在保持甚至提升迁移性能的同时,将可训练参数量减少了 88--99\%。