Deep learning has made significant progress in computer vision, specifically in image classification, object detection, and semantic segmentation. The skip connection has played an essential role in the architecture of deep neural networks,enabling easier optimization through residual learning during the training stage and improving accuracy during testing. Many neural networks have inherited the idea of residual learning with skip connections for various tasks, and it has been the standard choice for designing neural networks. This survey provides a comprehensive summary and outlook on the development of skip connections in deep neural networks. The short history of skip connections is outlined, and the development of residual learning in deep neural networks is surveyed. The effectiveness of skip connections in the training and testing stages is summarized, and future directions for using skip connections in residual learning are discussed. Finally, we summarize seminal papers, source code, models, and datasets that utilize skip connections in computer vision, including image classification, object detection, semantic segmentation, and image reconstruction. We hope this survey could inspire peer researchers in the community to develop further skip connections in various forms and tasks and the theory of residual learning in deep neural networks. The project page can be found at https://github.com/apple1986/Residual_Learning_For_Images

翻译：深度学习在计算机视觉领域取得了显著进展，特别是在图像分类、目标检测和语义分割方面。跳跃连接在深度神经网络架构中发挥了关键作用，通过训练阶段引入残差学习简化了优化过程，并提升了测试阶段的准确性。许多神经网络已将带跳跃连接的残差学习思想应用于各类任务，并成为设计神经网络的标准选择。本综述全面总结并展望了深度神经网络中跳跃连接的发展历程。首先概述了跳跃连接的简要历史，梳理了深度神经网络中残差学习的发展脉络，系统总结了跳跃连接在训练与测试阶段的有效性，并讨论了残差学习中跳跃连接的未来应用方向。最后，我们整理了计算机视觉（包括图像分类、目标检测、语义分割和图像重建）中采用跳跃连接的重要论文、源代码、模型及数据集。希望本综述能激励同行研究者进一步探索不同形式与任务下的跳跃连接，并推动深度神经网络中残差学习理论的发展。项目页面详见：https://github.com/apple1986/Residual_Learning_For_Images