Foundation models have indeed made a profound impact on various fields, emerging as pivotal components that significantly shape the capabilities of intelligent systems. In the context of intelligent vehicles, leveraging the power of foundation models has proven to be transformative, offering notable advancements in visual understanding. Equipped with multi-modal and multi-task learning capabilities, multi-modal multi-task visual understanding foundation models (MM-VUFMs) effectively process and fuse data from diverse modalities and simultaneously handle various driving-related tasks with powerful adaptability, contributing to a more holistic understanding of the surrounding scene. In this survey, we present a systematic analysis of MM-VUFMs specifically designed for road scenes. Our objective is not only to provide a comprehensive overview of common practices, referring to task-specific models, unified multi-modal models, unified multi-task models, and foundation model prompting techniques, but also to highlight their advanced capabilities in diverse learning paradigms. These paradigms include open-world understanding, efficient transfer for road scenes, continual learning, interactive and generative capability. Moreover, we provide insights into key challenges and future trends, such as closed-loop driving systems, interpretability, embodied driving agents, and world models. To facilitate researchers in staying abreast of the latest developments in MM-VUFMs for road scenes, we have established a continuously updated repository at https://github.com/rolsheng/MM-VUFM4DS

翻译：基础模型确实对各种领域产生了深远影响，成为塑造智能系统能力的关键组成部分。在智能车辆领域，利用基础模型的力量已被证明具有变革性，为视觉理解带来了显著进步。凭借多模态和多任务学习能力，多模态多任务视觉理解基础模型（MM-VUFMs）能够有效处理并融合来自不同模态的数据，同时以强大的适应性处理多种驾驶相关任务，从而促进对周围场景的更全面理解。本文综述中，我们对专门针对道路场景设计的MM-VUFMs进行了系统分析。我们的目标不仅在于全面概述常见实践，包括任务专用模型、统一多模态模型、统一多任务模型及基础模型提示技术，还在于强调它们在不同学习范式中的先进能力。这些范式包括开放世界理解、道路场景高效迁移、持续学习、交互与生成能力。此外，我们深入探讨了关键挑战与未来趋势，如闭环驾驶系统、可解释性、具身驾驶智能体以及世界模型。为帮助研究人员紧跟道路场景MM-VUFMs的最新进展，我们建立了一个持续更新的资源库，网址为https://github.com/rolsheng/MM-VUFM4DS。