A significant challenge in the field of object detection lies in the system's performance under non-ideal imaging conditions, such as rain, fog, low illumination, or raw Bayer images that lack ISP processing. Our study introduces "Feature Corrective Transfer Learning", a novel approach that leverages transfer learning and a bespoke loss function to facilitate the end-to-end detection of objects in these challenging scenarios without the need to convert non-ideal images into their RGB counterparts. In our methodology, we initially train a comprehensive model on a pristine RGB image dataset. Subsequently, non-ideal images are processed by comparing their feature maps against those from the initial ideal RGB model. This comparison employs the Extended Area Novel Structural Discrepancy Loss (EANSDL), a novel loss function designed to quantify similarities and integrate them into the detection loss. This approach refines the model's ability to perform object detection across varying conditions through direct feature map correction, encapsulating the essence of Feature Corrective Transfer Learning. Experimental validation on variants of the KITTI dataset demonstrates a significant improvement in mean Average Precision (mAP), resulting in a 3.8-8.1% relative enhancement in detection under non-ideal conditions compared to the baseline model, and a less marginal performance difference within 1.3% of the mAP@[0.5:0.95] achieved under ideal conditions by the standard Faster RCNN algorithm.
翻译:目标检测领域的一个重大挑战在于系统在非理想成像条件下的性能表现,例如雨、雾、低光照或缺乏ISP处理的原始Bayer图像。本研究提出"特征校正迁移学习"(Feature Corrective Transfer Learning)这一新颖方法,通过迁移学习和定制损失函数,实现这些挑战性场景中目标检测的端到端解决方案,且无需将非理想图像转换为RGB格式。在我们的方法中,首先在纯净RGB图像数据集上训练完整模型,随后通过将非理想图像的特征图与初始理想RGB模型的特征图进行比对来处理非理想图像。该比对采用扩展区域新型结构差异损失(EANSDL),这是一种用于量化相似性并将其整合到检测损失中的新型损失函数。通过直接特征图校正,该方法优化了模型在不同条件下的目标检测能力,体现了特征校正迁移学习的核心思想。在KITTI数据集变体上的实验验证表明,平均精度均值(mAP)显著提升,与基线模型相比,在非理想条件下检测性能相对提升3.8%-8.1%,且与标准Faster RCNN算法在理想条件下获得的mAP@[0.5:0.95]性能差距控制在1.3%以内。