Computer vision-based damage detection using remote cameras and unmanned aerial vehicles (UAVs) enables efficient and low-cost bridge health monitoring that reduces labor costs and the needs for sensor installation and maintenance. By leveraging recent semantic image segmentation approaches, we are able to find regions of critical structural components and recognize damage at the pixel level using images as the only input. However, existing methods perform poorly when detecting small damages (e.g., cracks and exposed rebars) and thin objects with limited image samples, especially when the components of interest are highly imbalanced. To this end, this paper introduces a semantic segmentation framework that imposes the hierarchical semantic relationship between component category and damage types. For example, certain concrete cracks only present on bridge columns and therefore the non-column region will be masked out when detecting such damages. In this way, the damage detection model could focus on learning features from possible damaged regions only and avoid the effects of other irrelevant regions. We also utilize multi-scale augmentation that provides views with different scales that preserves contextual information of each image without losing the ability of handling small and thin objects. Furthermore, the proposed framework employs important sampling that repeatedly samples images containing rare components (e.g., railway sleeper and exposed rebars) to provide more data samples, which addresses the imbalanced data challenge.

翻译：利用远程摄像头和无人机进行基于计算机视觉的损伤检测，能够实现高效低成本的桥梁健康监测，从而降低人力成本并减少传感器安装与维护需求。通过运用最新的语义图像分割方法，我们能够以图像作为唯一输入，在像素级别识别关键结构构件区域并检测损伤。然而，现有方法在检测小尺寸损伤（如裂缝和裸露钢筋）及图像样本有限的细长物体时表现不佳，尤其在目标构件存在严重类别不平衡的情况下。为此，本文提出一种语义分割框架，该框架通过构建构件类别与损伤类型之间的分层语义关系来提升检测性能。例如，某些混凝土裂缝仅出现在桥墩区域，因此在检测此类损伤时非桥墩区域将被屏蔽。通过这种方式，损伤检测模型能够专注于从可能受损区域学习特征，并避免其他无关区域的影响。我们还采用多尺度增强技术，通过提供不同尺度的图像视角，在保持每张图像上下文信息的同时不损失处理细小物体的能力。此外，所提框架采用重要性采样策略，通过重复采样包含稀有构件（如轨枕和裸露钢筋）的图像以提供更多数据样本，从而有效应对数据不平衡的挑战。