There is an increasing number of real-world problems in computer vision and machine learning requiring to take into consideration multiple interpretation layers (modalities or views) of the world and learn how they relate to each other. For example, in the case of Earth Observations from satellite data, it is important to be able to predict one observation layer (e.g. vegetation index) from other layers (e.g. water vapor, snow cover, temperature etc), in order to best understand how the Earth System functions and also be able to reliably predict information for one layer when the data is missing (e.g. due to measurement failure or error).
翻译:在计算机视觉与机器学习领域,越来越多的实际问题需要综合考虑世界的多重解释层(模态或视角),并学习它们之间的关联。例如,在基于卫星数据的地球观测中,必须能够从其他观测层(如水汽、雪盖、温度等)预测某一观测层(如植被指数),以便深入理解地球系统的运作机制,同时能够在数据缺失(例如因测量故障或误差)时可靠地预测该层的信息。