Finding target persons in full scene images with a query of text description has important practical applications in intelligent video surveillance.However, different from the real-world scenarios where the bounding boxes are not available, existing text-based person retrieval methods mainly focus on the cross modal matching between the query text descriptions and the gallery of cropped pedestrian images. To close the gap, we study the problem of text-based person search in full images by proposing a new end-to-end learning framework which jointly optimize the pedestrian detection, identification and visual-semantic feature embedding tasks. To take full advantage of the query text, the semantic features are leveraged to instruct the Region Proposal Network to pay more attention to the text-described proposals. Besides, a cross-scale visual-semantic embedding mechanism is utilized to improve the performance. To validate the proposed method, we collect and annotate two large-scale benchmark datasets based on the widely adopted image-based person search datasets CUHK-SYSU and PRW. Comprehensive experiments are conducted on the two datasets and compared with the baseline methods, our method achieves the state-of-the-art performance.

翻译：针对全场景图像中通过文本描述查询目标行人的问题，在智能视频监控领域具有重要实际应用价值。然而，现有文本行人检索方法主要聚焦于查询文本描述与裁剪行人图像库之间的跨模态匹配，这与现实中无法获得边界框的场景存在差异。为弥合这一差距，本文提出一种新型端到端学习框架，通过联合优化行人检测、身份识别及视觉-语义特征嵌入任务，研究全图文本行人搜索问题。为充分利用查询文本，语义特征被用于引导区域提案网络关注文本描述提案。此外，采用跨尺度视觉-语义嵌入机制提升性能。为验证方法有效性，基于广泛采用的图像行人搜索数据集CUHK-SYSU和PRW收集并标注了两个大规模基准数据集。综合实验表明，与基线方法相比，本方法在两个数据集上均取得了最先进性能。