Human intelligence effortlessly interprets visual scenes along a rich spectrum of semantic dimensions. However, existing approaches to language-grounded visual concept learning are limited to a few predefined primitive axes, such as color and shape, and are typically explored in synthetic datasets. In this work, we propose a scalable framework that adaptively identifies image-related concept axes and grounds visual concepts along these axes in real-world scenes. Leveraging a pretrained vision-language model and our universal prompting strategy, our framework identifies a diverse image-related axes without any prior knowledge. Our universal concept encoder adaptively binds visual features to the discovered axes without introducing additional model parameters for each concept. To ground visual concepts along the discovered axes, we optimize a compositional anchoring objective, which ensures that each axis can be independently manipulated without affecting others. We demonstrate the effectiveness of our framework on subsets of ImageNet, CelebA-HQ, and AFHQ, showcasing superior editing capabilities across diverse real-world concepts that are too varied to be manually predefined. Our method also exhibits strong compositional generalization, outperforming existing visual concept learning and text-based editing methods. The code is available at https://github.com/whieya/Language-grounded-VCL.

翻译：人类智能能够毫不费力地沿着丰富的语义维度解读视觉场景。然而，现有的语言锚定视觉概念学习方法仅限于少数预定义的原始轴（如颜色和形状），且通常在合成数据集中进行探索。本研究提出一种可扩展的框架，能够自适应地识别图像相关的概念轴，并在真实世界场景中沿这些轴锚定视觉概念。通过利用预训练的视觉-语言模型及我们提出的通用提示策略，该框架无需任何先验知识即可识别多样化的图像相关轴。我们的通用概念编码器能够自适应地将视觉特征绑定到已发现的轴上，而无需为每个概念引入额外的模型参数。为了沿已发现的轴锚定视觉概念，我们优化了组合锚定目标函数，确保每个轴可被独立操控而不影响其他轴。我们在ImageNet、CelebA-HQ和AFHQ的子集上验证了框架的有效性，展示了其处理多样化真实世界概念的卓越编辑能力——这些概念的多样性已超出人工预定义的范畴。本方法还展现出强大的组合泛化能力，其性能优于现有的视觉概念学习及基于文本的编辑方法。代码发布于https://github.com/whieya/Language-grounded-VCL。