Machine vision is susceptible to laser dazzle, where intense laser light can blind and distort its perception of the environment through oversaturation or permanent damage to sensor pixels. Here we employ a wavefront-coded phase mask to diffuse the energy of laser light and introduce a sandwich generative adversarial network (SGAN) to restore images from complex image degradations, such as varying laser-induced image saturation, mask-induced image blurring, unknown lighting conditions, and various noise corruptions. The SGAN architecture combines discriminative and generative methods by wrapping two GANs around a learnable image deconvolution module. In addition, we make use of Fourier feature representations to reduce the spectral bias of neural networks and improve its learning of high-frequency image details. End-to-end training includes the realistic physics-based synthesis of a large set of training data from publicly available images. We trained the SGAN to suppress the peak laser irradiance as high as $10^6$ times the sensor saturation threshold - the point at which camera sensors may experience damage without the mask. The trained model was evaluated on both a synthetic data set and data collected from the laboratory. The proposed image restoration model quantitatively and qualitatively outperforms state-of-the-art methods for a wide range of scene contents, laser powers, incident laser angles, ambient illumination strengths, and noise characteristics.
翻译:机器视觉易受激光炫光影响,强激光可通过过度饱和或永久损坏传感器像素,导致其失明并扭曲环境感知。本文采用波前编码相位掩模扩散激光能量,并引入夹层生成对抗网络(SGAN)以恢复由激光诱导的复杂图像退化,包括不同强度的激光饱和、掩模引起的图像模糊、未知光照条件及多种噪声干扰。SGAN架构融合了判别式与生成式方法,通过将两个GAN网络包裹于可学习的图像去卷积模块周围实现。此外,我们利用傅里叶特征表示降低神经网络的频谱偏差,提升其对图像高频细节的学习能力。端到端训练包含基于物理原理的合成方法,可从公开图像生成大规模训练数据集。我们训练的SGAN能够抑制高达传感器饱和阈值$10^6$倍的峰值激光辐照度——在该强度下,若无掩模保护,相机传感器可能遭受损坏。该模型在合成数据集与实验室采集数据上均进行了评估。在广泛场景内容、激光功率、入射角度、环境光照强度及噪声特性下,所提出的图像恢复模型在定量与定性上均优于现有最佳方法。