Smart Digital twins (SDTs) are being increasingly used to virtually replicate and predict the behaviors of complex physical systems through continual data assimilation enabling the optimization of the performance of these systems by controlling the actions of systems. Recently, deep learning (DL) models have significantly enhanced the capabilities of SDTs, particularly for tasks such as predictive maintenance, anomaly detection, and optimization. In many domains, including medicine, engineering, and education, SDTs use image data (image-based SDTs) to observe and learn system behaviors and control their behaviors. This paper focuses on various approaches and associated challenges in developing image-based SDTs by continually assimilating image data from physical systems. The paper also discusses the challenges involved in designing and implementing DL models for SDTs, including data acquisition, processing, and interpretation. In addition, insights into the future directions and opportunities for developing new image-based DL approaches to develop robust SDTs are provided. This includes the potential for using generative models for data augmentation, developing multi-modal DL models, and exploring the integration of DL with other technologies, including 5G, edge computing, and IoT. In this paper, we describe the image-based SDTs, which enable broader adoption of the digital twin DT paradigms across a broad spectrum of areas and the development of new methods to improve the abilities of SDTs in replicating, predicting, and optimizing the behavior of complex systems.

翻译：智能数字孪生（SDT）正越来越多地用于通过持续数据同化虚拟复制和预测复杂物理系统的行为，从而通过控制系统动作优化其性能。近年来，深度学习（DL）模型显著增强了SDT的能力，特别是在预测性维护、异常检测和优化等任务中。在医学、工程和教育等多个领域，SDT利用图像数据（基于图像的SDT）来观察和学习系统行为并控制其行为。本文重点探讨了通过持续同化物理系统图像数据开发基于图像的SDT的各种方法及相关挑战。文章还讨论了为SDT设计和实现深度学习模型所涉及的挑战，包括数据采集、处理和解释。此外，本文为开发新型基于图像的深度学习方法以构建稳健的SDT提供了未来方向与机遇的洞察，包括利用生成模型进行数据增强、开发多模态深度学习模型，以及探索深度学习与5G、边缘计算和物联网等其他技术的融合潜力。本文阐述了基于图像的SDT，这些技术使得数字孪生（DT）范式能够在更广泛领域中推广应用，并促进了提升SDT在复制、预测和优化复杂系统行为方面能力的新方法开发。