Eliminating the rain degradation in stereo images poses a formidable challenge, which necessitates the efficient exploitation of mutual information present between the dual views. To this end, we devise MQINet, which employs multi-dimension queries and interactions for stereo image deraining. More specifically, our approach incorporates a context-aware dimension-wise queried block (CDQB). This module leverages dimension-wise queries that are independent of the input features and employs global context-aware attention (GCA) to capture essential features while avoiding the entanglement of redundant or irrelevant information. Meanwhile, we introduce an intra-view physics-aware attention (IPA) based on the inverse physical model of rainy images. IPA extracts shallow features that are sensitive to the physics of rain degradation, facilitating the reduction of rain-related artifacts during the early learning period. Furthermore, we integrate a cross-view multi-dimension interacting attention mechanism (CMIA) to foster comprehensive feature interaction between the two views across multiple dimensions. Extensive experimental evaluations demonstrate the superiority of our model over EPRRNet and StereoIRR, achieving respective improvements of 4.18 dB and 0.45 dB in PSNR. Code and models are available at \url{https://github.com/chdwyb/MQINet}.
翻译:消除立体图像中的雨雪退化构成了一项严峻挑战,这需要有效利用双视图之间存在的互信息。为此,我们设计了MQINet,该网络采用多维度查询与交互机制进行立体图像去雨。具体而言,我们的方法包含一个上下文感知的维度查询块(CDQB)。该模块利用独立于输入特征的维度查询,并采用全局上下文感知注意力(GCA)来捕获关键特征,同时避免冗余或无关信息的纠缠。同时,我们引入了一种基于雨天图像逆物理模型的视图内物理感知注意力(IPA)。IPA提取对雨雪退化物理特性敏感的浅层特征,有助于在早期学习阶段减少与雨雪相关的伪影。此外,我们集成了一种跨视图多维度交互注意力机制(CMIA),以促进两个视图之间跨多个维度的全面特征交互。大量实验评估表明,我们的模型优于EPRRNet和StereoIRR,在PSNR上分别实现了4.18 dB和0.45 dB的提升。代码和模型可在\url{https://github.com/chdwyb/MQINet}获取。