This report serves as a supplementary document for TaskPrompter, detailing its implementation on a new joint 2D-3D multi-task learning benchmark based on Cityscapes-3D. TaskPrompter presents an innovative multi-task prompting framework that unifies the learning of (i) task-generic representations, (ii) task-specific representations, and (iii) cross-task interactions, as opposed to previous approaches that separate these learning objectives into different network modules. This unified approach not only reduces the need for meticulous empirical structure design but also significantly enhances the multi-task network's representation learning capability, as the entire model capacity is devoted to optimizing the three objectives simultaneously. TaskPrompter introduces a new multi-task benchmark based on Cityscapes-3D dataset, which requires the multi-task model to concurrently generate predictions for monocular 3D vehicle detection, semantic segmentation, and monocular depth estimation. These tasks are essential for achieving a joint 2D-3D understanding of visual scenes, particularly in the development of autonomous driving systems. On this challenging benchmark, our multi-task model demonstrates strong performance compared to single-task state-of-the-art methods and establishes new state-of-the-art results on the challenging 3D detection and depth estimation tasks.

翻译：本报告作为TaskPrompter的补充文档，详细阐述了其在基于Cityscapes-3D的新型联合2D-3D多任务学习基准上的实现。TaskPrompter提出了一种创新的多任务提示框架，统一了（i）任务通用表征、（ii）任务特定表征和（iii）跨任务交互的学习过程，与传统方法将这些学习目标分离到不同网络模块的做法形成对比。这种统一方法不仅减少了精细经验结构设计的需求，还显著增强了多任务网络的表征学习能力，因为整个模型容量被用于同时优化这三个目标。TaskPrompter基于Cityscapes-3D数据集引入了一个新的多任务基准，要求多任务模型同时生成单目3D车辆检测、语义分割和单目深度估计的预测。这些任务对于实现视觉场景的联合2D-3D理解至关重要，尤其在自动驾驶系统的开发中。在这个具有挑战性的基准上，我们的多任务模型相较于单任务最先进方法展现出强劲性能，并在具有挑战性的3D检测和深度估计任务上取得了新的最先进成果。