Extracting useful visual cues for the downstream tasks is especially challenging under low-light vision. Prior works create enhanced representations by either correlating visual quality with machine perception or designing illumination-degrading transformation methods that require pre-training on synthetic datasets. We argue that optimizing enhanced image representation pertaining to the loss of the downstream task can result in more expressive representations. Therefore, in this work, we propose a novel module, FeatEnHancer, that hierarchically combines multiscale features using multiheaded attention guided by task-related loss function to create suitable representations. Furthermore, our intra-scale enhancement improves the quality of features extracted at each scale or level, as well as combines features from different scales in a way that reflects their relative importance for the task at hand. FeatEnHancer is a general-purpose plug-and-play module and can be incorporated into any low-light vision pipeline. We show with extensive experimentation that the enhanced representation produced with FeatEnHancer significantly and consistently improves results in several low-light vision tasks, including dark object detection (+5.7 mAP on ExDark), face detection (+1.5 mAPon DARK FACE), nighttime semantic segmentation (+5.1 mIoU on ACDC ), and video object detection (+1.8 mAP on DarkVision), highlighting the effectiveness of enhancing hierarchical features under low-light vision.
翻译:在弱光视觉条件下,为下游任务提取有效视觉线索尤为困难。现有工作通过关联视觉质量与机器感知,或设计需要依赖合成数据集预训练的照明退化变换方法生成增强表示。我们认为,针对下游任务损失函数优化增强图像表示能够产生更具表达力的特征。为此,本文提出新型模块FeatEnHancer,该模块通过任务相关损失函数引导的多头注意力机制,层次化融合多尺度特征以创建适宜表示。此外,我们的尺度内增强机制不仅提升了各尺度层级特征质量,还以反映其对当前任务相对重要性的方式组合不同尺度的特征。FeatEnHancer作为通用即插即用模块,可无缝集成至任意弱光视觉处理流程。大量实验表明,经FeatEnHancer处理的增强表示在多项弱光视觉任务中取得显著且一致的性能提升,包括暗光目标检测(ExDark上+5.7 mAP)、人脸检测(DARK FACE上+1.5 mAP)、夜间语义分割(ACDC上+5.1 mIoU)及视频目标检测(DarkVision上+1.8 mAP),充分验证了弱光视觉下层次化特征增强方法的有效性。