Multimodal medical data fusion has emerged as a transformative approach in smart healthcare, enabling a comprehensive understanding of patient health and personalized treatment plans. In this paper, a journey from data, information, and knowledge to wisdom (DIKW) is explored through multimodal fusion for smart healthcare. A comprehensive review of multimodal medical data fusion focuses on the integration of various data modalities are presented. It explores different approaches such as Feature selection, Rule-based systems, Machine learning, Deep learning, and Natural Language Processing for fusing and analyzing multimodal data. The paper also highlights the challenges associated with multimodal fusion in healthcare. By synthesizing the reviewed frameworks and insights, a generic framework for multimodal medical data fusion is proposed while aligning with the DIKW mechanism. Moreover, it discusses future directions aligned with the four pillars of healthcare: Predictive, Preventive, Personalized, and Participatory approaches based on the DIKW and the generic framework. The components from this comprehensive survey form the foundation for the successful implementation of multimodal fusion in smart healthcare. The findings of this survey can guide researchers and practitioners in leveraging the power of multimodal fusion with the approaches to revolutionize healthcare and improve patient outcomes.
翻译:多模态医疗数据融合已成为智慧医疗领域的一项变革性方法,能够全面理解患者健康状况并制定个性化治疗方案。本文通过多模态融合,探索了从数据、信息、知识到智慧(DIKW)的旅程。研究对多模态医疗数据融合进行了全面综述,重点探讨了不同数据模态的集成方法。本文分析了多种融合与分析多模态数据的技术路径,包括特征选择、规则系统、机器学习、深度学习以及自然语言处理。同时,论文也强调了医疗领域多模态融合面临的挑战。通过综合梳理现有框架与洞见,本文提出了一个与DIKW机制相契合的多模态医疗数据融合通用框架。此外,基于DIKW框架与通用框架,本文讨论了与医疗四大支柱(预测性、预防性、个性化与参与性方法)相一致的未来发展方向。本综述中的各个组成部分为多模态融合在智慧医疗中的成功实施奠定了基础。其研究结果可为研究人员和实践者提供指导,帮助他们借助相关方法发挥多模态融合的力量,推动医疗变革并改善患者预后。