Finding target persons in full scene images with a query of text description has important practical applications in intelligent video surveillance.However, different from the real-world scenarios where the bounding boxes are not available, existing text-based person retrieval methods mainly focus on the cross modal matching between the query text descriptions and the gallery of cropped pedestrian images. To close the gap, we study the problem of text-based person search in full images by proposing a new end-to-end learning framework which jointly optimize the pedestrian detection, identification and visual-semantic feature embedding tasks. To take full advantage of the query text, the semantic features are leveraged to instruct the Region Proposal Network to pay more attention to the text-described proposals. Besides, a cross-scale visual-semantic embedding mechanism is utilized to improve the performance. To validate the proposed method, we collect and annotate two large-scale benchmark datasets based on the widely adopted image-based person search datasets CUHK-SYSU and PRW. Comprehensive experiments are conducted on the two datasets and compared with the baseline methods, our method achieves the state-of-the-art performance.

翻译：全场景图像中通过文本描述查询目标人物在智能视频监控中具有重要应用价值。然而，与真实场景中无边界框的现实条件不同，现有基于文本的行人检索方法主要聚焦于查询文本描述与裁剪行人图像库之间的跨模态匹配。为弥合这一差距，我们通过提出全新的端到端学习框架，联合优化行人检测、身份识别及视觉-语义特征嵌入任务，研究全图像中的文本行人搜索问题。为充分利用查询文本，我们利用语义特征引导区域候选网络（Region Proposal Network）更关注文本描述的候选框。此外，采用跨尺度视觉-语义嵌入机制提升性能。为验证所提方法，我们基于广泛采用的图像行人搜索数据集CUHK-SYSU和PRW收集并标注了两个大规模基准数据集。在两个数据集上的综合实验表明，与基线方法相比，我们的方法达到了最先进的性能。