In this paper, we present GyroDeblurNet, a novel single image deblurring method that utilizes a gyro sensor to effectively resolve the ill-posedness of image deblurring. The gyro sensor provides valuable information about camera motion during exposure time that can significantly improve deblurring quality. However, effectively exploiting real-world gyro data is challenging due to significant errors from various sources including sensor noise, the disparity between the positions of a camera module and a gyro sensor, the absence of translational motion information, and moving objects whose motions cannot be captured by a gyro sensor. To handle gyro error, GyroDeblurNet is equipped with two novel neural network blocks: a gyro refinement block and a gyro deblurring block. The gyro refinement block refines the error-ridden gyro data using the blur information from the input image. On the other hand, the gyro deblurring block removes blur from the input image using the refined gyro data and further compensates for gyro error by leveraging the blur information from the input image. For training a neural network with erroneous gyro data, we propose a training strategy based on the curriculum learning. We also introduce a novel gyro data embedding scheme to represent real-world intricate camera shakes. Finally, we present a synthetic dataset and a real dataset for the training and evaluation of gyro-based single image deblurring. Our experiments demonstrate that our approach achieves state-of-the-art deblurring quality by effectively utilizing erroneous gyro data.

翻译：在本文中，我们提出GyroDeblurNet，一种利用陀螺仪传感器有效解决图像去模糊不适定性的新型单图像去模糊方法。陀螺仪传感器在曝光时间内提供关于相机运动的有价值信息，可显著改善去模糊质量。然而，由于传感器噪声、相机模块与陀螺仪传感器之间的位置差异、平移运动信息的缺失以及陀螺仪无法捕捉的运动物体等多种来源的显著误差，有效利用真实世界的陀螺仪数据极具挑战性。为处理陀螺仪误差，GyroDeblurNet配备了两种新型神经网络模块：陀螺仪精化模块和陀螺仪去模糊模块。陀螺仪精化模块利用输入图像中的模糊信息修正含噪的陀螺仪数据；而陀螺仪去模糊模块则利用精化后的陀螺仪数据去除输入图像中的模糊，并进一步通过输入图像的模糊信息补偿陀螺仪误差。针对含噪陀螺仪数据的神经网络训练，我们提出基于课程学习的训练策略，同时引入新型陀螺仪数据嵌入方案以表征真实世界中复杂的相机抖动。最后，我们构建了用于陀螺仪单图像去模糊训练与评估的合成数据集和真实数据集。实验表明，通过有效利用含噪陀螺仪数据，本方法达到了最先进的去模糊质量。