The recent large-scale Contrastive Language-Image Pretraining (CLIP) model has shown great potential in various downstream tasks via leveraging the pretrained vision and language knowledge. Scene text, which contains rich textual and visual information, has an inherent connection with a model like CLIP. Recently, pretraining approaches based on vision language models have made effective progresses in the field of text detection. In contrast to these works, this paper proposes a new method, termed TCM, focusing on Turning the CLIP Model directly for text detection without pretraining process. We demonstrate the advantages of the proposed TCM as follows: (1) The underlying principle of our framework can be applied to improve existing scene text detector. (2) It facilitates the few-shot training capability of existing methods, e.g., by using 10% of labeled data, we significantly improve the performance of the baseline method with an average of 22% in terms of the F-measure on 4 benchmarks. (3) By turning the CLIP model into existing scene text detection methods, we further achieve promising domain adaptation ability. The code will be publicly released at https://github.com/wenwenyu/TCM.

翻译：近期大规模对比语言-图像预训练（CLIP）模型通过利用预训练的视觉与语言知识，在多种下游任务中展现出巨大潜力。场景文本因包含丰富的文本与视觉信息，与CLIP这类模型存在内在关联。基于视觉语言模型的预训练方法已在文本检测领域取得有效进展。与这些工作不同，本文提出一种名为TCM的新方法，专注于无需预训练过程即可直接转化CLIP模型用于文本检测。我们论证了所提TCM方法的以下优势：（1）本框架的基本原理可应用于改进现有场景文本检测器；（2）它增强了现有方法的小样本训练能力，例如仅使用10%标注数据，我们在4个基准数据集上使基线方法的F值平均提升22%；（3）通过将CLIP模型转化为现有场景文本检测方法，我们进一步实现了有前景的领域自适应能力。相关代码将在https://github.com/wenwenyu/TCM公开发布。