Acquiring the desired font for various design tasks can be challenging and requires professional typographic knowledge. While previous font retrieval or generation works have alleviated some of these difficulties, they often lack support for multiple languages and semantic attributes beyond the training data domains. To solve this problem, we present FontCLIP: a model that connects the semantic understanding of a large vision-language model with typographical knowledge. We integrate typography-specific knowledge into the comprehensive vision-language knowledge of a pretrained CLIP model through a novel finetuning approach. We propose to use a compound descriptive prompt that encapsulates adaptively sampled attributes from a font attribute dataset focusing on Roman alphabet characters. FontCLIP's semantic typographic latent space demonstrates two unprecedented generalization abilities. First, FontCLIP generalizes to different languages including Chinese, Japanese, and Korean (CJK), capturing the typographical features of fonts across different languages, even though it was only finetuned using fonts of Roman characters. Second, FontCLIP can recognize the semantic attributes that are not presented in the training data. FontCLIP's dual-modality and generalization abilities enable multilingual and cross-lingual font retrieval and letter shape optimization, reducing the burden of obtaining desired fonts.

翻译：获取设计任务中所需的字体可能颇具挑战，且需要专业的印刷知识。尽管以往的字体检索或生成工作已在一定程度上缓解了这些困难，但它们往往缺乏对多语言的支持以及对训练数据域外语义属性的泛化能力。为解决这一问题，我们提出了FontCLIP：一种将大规模视觉-语言模型的语义理解能力与印刷知识相连接的模型。我们通过一种新颖的微调方法，将印刷领域的特定知识整合到预训练CLIP模型全面的视觉-语言知识中。我们提出使用复合描述性提示，该提示封装了从聚焦于罗马字母字符的字体属性数据集中自适应采样的属性。FontCLIP的语义印刷潜在空间展示了两种前所未有的泛化能力。首先，FontCLIP能够泛化到包括中文、日文和韩文（CJK）在内的不同语言，捕捉跨语言字体的印刷特征，尽管其仅使用罗马字符字体进行微调。其次，FontCLIP可识别训练数据中未出现的语义属性。FontCLIP的双模态与泛化能力使其能够实现多语言及跨语言的字体检索与字母形状优化，从而减轻获取所需字体的负担。