The recent large-scale Contrastive Language-Image Pretraining (CLIP) model has shown great potential in various downstream tasks via leveraging the pretrained vision and language knowledge. Scene text, which contains rich textual and visual information, has an inherent connection with a model like CLIP. Recently, pretraining approaches based on vision language models have made effective progresses in the field of text detection. In contrast to these works, this paper proposes a new method, termed TCM, focusing on Turning the CLIP Model directly for text detection without pretraining process. We demonstrate the advantages of the proposed TCM as follows: (1) The underlying principle of our framework can be applied to improve existing scene text detector. (2) It facilitates the few-shot training capability of existing methods, e.g., by using 10% of labeled data, we significantly improve the performance of the baseline method with an average of 22% in terms of the F-measure on 4 benchmarks. (3) By turning the CLIP model into existing scene text detection methods, we further achieve promising domain adaptation ability. The code will be publicly released.

翻译：近期大规模对比语言-图像预训练（CLIP）模型通过利用预训练的视觉与语言知识，在各类下游任务中展现出巨大潜力。场景文字因其包含丰富的文本与视觉信息，与CLIP这类模型存在内在关联。基于视觉语言模型的预训练方法近年来在文字检测领域取得了有效进展。与此类工作不同，本文提出一种名为TCM的新方法，专注于直接转化CLIP模型进行文字检测，无需预训练过程。我们论证了所提TCM方法的如下优势：（1）本框架的基本原理可应用于改进现有场景文字检测器。（2）该方法增强了现有方法的少样本训练能力，例如仅使用10%标注数据，我们即在4个基准测试中使基线方法的F值平均提升22%。（3）通过将CLIP模型融入现有场景文字检测方法，我们进一步获得了优越的领域自适应能力。相关代码将公开发布。