In the field of autonomous driving, a variety of sensor data types exist, each representing different modalities of the same scene. Therefore, it is feasible to utilize data from other sensors to facilitate image compression. However, few techniques have explored the potential benefits of utilizing inter-modality correlations to enhance the image compression performance. In this paper, motivated by the recent success of learned image compression, we propose a new framework that uses sparse point clouds to assist in learned image compression in the autonomous driving scenario. We first project the 3D sparse point cloud onto a 2D plane, resulting in a sparse depth map. Utilizing this depth map, we proceed to predict camera images. Subsequently, we use these predicted images to extract multi-scale structural features. These features are then incorporated into learned image compression pipeline as additional information to improve the compression performance. Our proposed framework is compatible with various mainstream learned image compression models, and we validate our approach using different existing image compression methods. The experimental results show that incorporating point cloud assistance into the compression pipeline consistently enhances the performance.

翻译：在自动驾驶领域，存在多种传感器数据类型，每种数据都表征同一场景的不同模态。因此，利用来自其他传感器的数据来辅助图像压缩是可行的。然而，目前鲜有技术探索利用模态间相关性来提升图像压缩性能的潜在优势。本文受近年来深度学习图像压缩成功的启发，提出了一种在自动驾驶场景中利用稀疏点云辅助深度学习图像压缩的新框架。我们首先将三维稀疏点云投影到二维平面，生成稀疏深度图。利用该深度图，我们进而预测相机图像。随后，我们使用这些预测图像提取多尺度结构特征。这些特征作为附加信息被整合到深度学习图像压缩流程中，以提升压缩性能。我们提出的框架兼容多种主流的深度学习图像压缩模型，并通过不同的现有图像压缩方法验证了我们的方案。实验结果表明，在压缩流程中引入点云辅助能持续提升性能。