Crowdsourced platforms provide huge amounts of street-view images that contain valuable building information. This work addresses the challenges in applying Scene Text Recognition (STR) in crowdsourced street-view images for building attribute mapping. We use Flickr images, particularly examining texts on building facades. A Berlin Flickr dataset is created, and pre-trained STR models are used for text detection and recognition. Manual checking on a subset of STR-recognized images demonstrates high accuracy. We examined the correlation between STR results and building functions, and analysed instances where texts were recognized on residential buildings but not on commercial ones. Further investigation revealed significant challenges associated with this task, including small text regions in street-view images, the absence of ground truth labels, and mismatches in buildings in Flickr images and building footprints in OpenStreetMap (OSM). To develop city-wide mapping beyond urban hotspot locations, we suggest differentiating the scenarios where STR proves effective while developing appropriate algorithms or bringing in additional data for handling other cases. Furthermore, interdisciplinary collaboration should be undertaken to understand the motivation behind building photography and labeling. The STR-on-Flickr results are publicly available at https://github.com/ya0-sun/STR-Berlin.
翻译:众包平台提供了大量包含宝贵建筑信息的街景图像。本研究探讨了在众包街景图像中应用场景文本识别(STR)进行建筑属性映射所面临的挑战。我们使用Flickr图像,重点识别建筑外墙上的文本。创建了柏林Flickr数据集,并利用预训练STR模型进行文本检测与识别。对STR识别结果子集的人工核查证实了高准确性。我们分析了STR结果与建筑功能之间的相关性,并研究了住宅建筑上识别到文本而商业建筑上未识别到的实例。进一步调查揭示了该任务面临的显著挑战,包括街景图像中文本区域过小、缺乏地面真值标签,以及Flickr图像中的建筑与OpenStreetMap(OSM)建筑轮廓不匹配。为实现城市热点区域之外的全市范围映射,我们建议区分STR有效的场景,同时开发适当算法或引入额外数据以处理其他情况。此外,应开展跨学科合作以理解建筑摄影与标注的动机。STR-on-Flickr结果已公开于https://github.com/ya0-sun/STR-Berlin。