Snow removal aims to locate snow areas and recover clean images without repairing traces. Unlike the regularity and semitransparency of rain, snow with various patterns and degradations seriously occludes the background. As a result, the state-of-the-art snow removal methods usually retains a large parameter size. In this paper, we propose a lightweight but high-efficient snow removal network called Laplace Mask Query Transformer (LMQFormer). Firstly, we present a Laplace-VQVAE to generate a coarse mask as prior knowledge of snow. Instead of using the mask in dataset, we aim at reducing both the information entropy of snow and the computational cost of recovery. Secondly, we design a Mask Query Transformer (MQFormer) to remove snow with the coarse mask, where we use two parallel encoders and a hybrid decoder to learn extensive snow features under lightweight requirements. Thirdly, we develop a Duplicated Mask Query Attention (DMQA) that converts the coarse mask into a specific number of queries, which constraint the attention areas of MQFormer with reduced parameters. Experimental results in popular datasets have demonstrated the efficiency of our proposed model, which achieves the state-of-the-art snow removal quality with significantly reduced parameters and the lowest running time.
翻译:除雪旨在定位雪区域并恢复无修复痕迹的清洁图像。与降水的规律性和半透明性不同,具有多种模式和退化的雪严重遮挡了背景。因此,最先进的除雪方法通常保留较大的参数规模。本文提出一种轻量级且高效的除雪网络,称为拉普拉斯掩码查询Transformer(LMQFormer)。首先,我们提出一种拉普拉斯-VQVAE来生成粗略掩码作为雪的先验知识。并非使用数据集中的掩码,我们旨在减少雪的信息熵以及恢复的计算成本。其次,我们设计了一种掩码查询Transformer(MQFormer)来利用粗略掩码进行除雪,其中使用两个并行编码器和一个混合解码器在轻量化要求下学习广泛的雪特征。第三,我们开发了一种重复掩码查询注意力(DMQA),将粗略掩码转换为特定数量的查询,从而约束MQFormer的注意力区域并减少参数。在流行数据集上的实验结果表明,本文提出的模型具有高效性,以显著减少的参数和最低运行时间实现了最先进的除雪质量。