The advent of large vision-language models (LVLMs) represents a noteworthy advancement towards the pursuit of artificial general intelligence. However, the extent of their efficacy across both specialized and general tasks warrants further investigation. This article endeavors to evaluate the competency of popular LVLMs in specialized and general tasks, respectively, aiming to offer a comprehensive comprehension of these innovative methodologies. To gauge their efficacy in specialized tasks, we tailor a comprehensive testbed comprising three distinct scenarios: natural, healthcare, and industrial, encompassing six challenging tasks. These tasks include salient, camouflaged, and transparent object detection, as well as polyp and skin lesion detection, alongside industrial anomaly detection. We examine the performance of three recent open-source LVLMs -- MiniGPT-v2, LLaVA-1.5, and Shikra -- in the realm of visual recognition and localization. Moreover, we conduct empirical investigations utilizing the aforementioned models alongside GPT-4V, assessing their multi-modal understanding capacities in general tasks such as object counting, absurd question answering, affordance reasoning, attribute recognition, and spatial relation reasoning. Our investigations reveal that these models demonstrate limited proficiency not only in specialized tasks but also in general tasks. We delve deeper into this inadequacy and suggest several potential factors, including limited cognition in specialized tasks, object hallucination, text-to-image interference, and decreased robustness in complex problems. We hope this study would provide valuable insights for the future development of LVLMs, augmenting their power in coping with both general and specialized applications.

翻译：大型视觉语言模型的出现代表了向通用人工智能追求的重要进展。然而，这些模型在专业任务和通用任务中的效能范围仍有待进一步探究。本文旨在分别评估主流大型视觉语言模型在专业任务和通用任务中的能力，以期对这些创新方法提供全面理解。为衡量其在专业任务中的效能，我们设计了一个涵盖自然场景、医疗保健和工业三个不同领域的综合测试平台，包含六项挑战性任务：显著目标检测、伪装目标检测、透明目标检测、息肉检测、皮肤病变检测以及工业异常检测。我们评估了三种近期开源大型视觉语言模型——MiniGPT-v2、LLaVA-1.5和Shikra——在视觉识别与定位方面的表现。此外，我们利用上述模型及GPT-4V进行了实证研究，评估它们在通用任务中的多模态理解能力，包括目标计数、荒谬问题回答、可供性推理、属性识别及空间关系推理。研究揭示，这些模型不仅在专业任务中表现有限，在通用任务中也同样存在不足。我们深入分析了这一缺陷，并提出了若干潜在因素，包括专业任务中的认知局限、目标幻觉、文本-图像干扰以及复杂问题中的鲁棒性下降。我们希望本研究能为大型视觉语言模型的未来发展提供有价值的见解，增强其应对通用及专业应用的能力。