Multimodal medical data fusion has emerged as a transformative approach in smart healthcare, enabling a comprehensive understanding of patient health and personalized treatment plans. In this paper, a journey from data, information, and knowledge to wisdom (DIKW) is explored through multimodal fusion for smart healthcare. A comprehensive review of multimodal medical data fusion focuses on the integration of various data modalities are presented. It explores different approaches such as Feature selection, Rule-based systems, Machine learning, Deep learning, and Natural Language Processing for fusing and analyzing multimodal data. The paper also highlights the challenges associated with multimodal fusion in healthcare. By synthesizing the reviewed frameworks and insights, a generic framework for multimodal medical data fusion is proposed while aligning with the DIKW mechanism. Moreover, it discusses future directions aligned with the four pillars of healthcare: Predictive, Preventive, Personalized, and Participatory approaches based on the DIKW and the generic framework. The components from this comprehensive survey form the foundation for the successful implementation of multimodal fusion in smart healthcare. The findings of this survey can guide researchers and practitioners in leveraging the power of multimodal fusion with the approaches to revolutionize healthcare and improve patient outcomes.
翻译:多模态医疗数据融合已成为智慧医疗中的一项变革性方法,能够全面理解患者健康状况并制定个性化治疗方案。本文通过智慧医疗中的多模态融合,探索了从数据、信息、知识到智慧(DIKW)的旅程。我们对多模态医疗数据融合进行了全面综述,重点关注不同数据模态的集成。文章探讨了特征选择、基于规则的系统、机器学习、深度学习以及自然语言处理等多种方法,用于融合与分析多模态数据。同时,本文还强调了多模态融合在医疗领域面临的挑战。通过综合所综述的框架与见解,我们提出了一种与DIKW机制相一致的通用多模态医疗数据融合框架。此外,文章基于DIKW机制与该通用框架,讨论了与医疗四大支柱(预测性、预防性、个性化与参与性方法)相一致的未来发展方向。本综述的要素为智慧医疗中多模态融合的成功实施奠定了基础。其研究结果可指导研究人员和实践者利用多模态融合的力量及相关方法,推动医疗行业变革并改善患者预后。