The emergence of pretrained models has significantly impacted Natural Language Processing (NLP) and Computer Vision to relational datasets. Traditionally, these models are assessed through fine-tuned downstream tasks. However, this raises the question of how to evaluate these models more efficiently and more effectively. In this study, we explore a novel approach where we leverage the meta features associated with each entity as a source of worldly knowledge and employ entity representations from the models. We propose using the consistency between these representations and the meta features as a metric for evaluating pretrained models. Our method's effectiveness is demonstrated across various domains, including models with relational datasets, large language models and image models.

翻译：预训练模型的出现显著影响了自然语言处理（NLP）和计算机视觉向关系数据集的迁移。传统上，这些模型通过微调的下游任务进行评估。然而，这引发了一个问题：如何更高效、更有效地评估这些模型？在本研究中，我们探索了一种新颖的方法，利用与每个实体相关联的元特征作为世界知识来源，并采用模型中的实体表示。我们提出将这些表示与元特征之间的一致性作为评估预训练模型的指标。我们的方法在多个领域展示了有效性，包括关系数据集模型、大型语言模型和图像模型。